The PC Informant » Search

Searching for meaning

Vic Laurie — Tue, 17 Nov 2009 10:24:46 +0000

The type of Internet search that we have now is mostly based on scanning for specific text although searching pictures is beginning to add some graphical features. The present search algorithms work remarkably well but have serious limitations and often turn up irrelevant links.

For example, if you ask a question, search engines will be unaware of the context and will instead look for pages containing the specific words used in the question. So if your search is, “How many lawyers work at Microsoft”, the computers will give you links to pages containing “Microsoft”, “lawyers”, and “work” but will not answer your question. I tried it on Google and got 2,650,000 hits. None of the first few hundred links answered the question although one estimated the number of lawyers at Disney. The answer may have been buried somewhere in the hundreds of thousands of pages but as a practical matter the question was not answered. When I searched the exact phrase, “How many lawyers work at Microsoft”, Google said nothing was found. I tried “size of Microsoft legal staff ” but had no luck. Bing didn’t know either.

Then I tried “legal budget Microsoft”. There were 7,270,000 hits but the answer was actually in the very first one. (If you are curious, the answer is an annual legal budget of $900 million and 1050 employees in its legal department, including 450 attorneys.) So, if you try different phrases, synonyms, and related subjects you may finally get your answer.

The example I have just given illustrates some of the problems with search algorithms but at other times the results are remarkably good. I searched Google for the question, “How many people live in Schenectady NY”, and got the answer right at the top of the search listing. But it was the answer for the county, not the city. The city population was several entries down. Bing gave the city population in the third entry.

Ideally, the computer would understand the meaning and context of our search queries but that’s a tall order and asks a lot of computers. Still, the goal of semantic search is being pursued and progress is being made. Science Daily reports a recent development:

European researchers have created the first integrated semantic search platform that integrates text, video and audio. The system can ‘watch’ films, ‘listen’ to audio and ‘read’ text to find relevant responses to semantic search terms. At last, computers are able to look for meaning in our multimedia searches.

Further on, the article discusses semantic search:

Right now, text in computing is defined by a series of numbers, most commonly the Unicode standard. Each number signifies a particular letter, and computers can scan these codes very quickly. So when you enter a search term, the machine has no idea what those letters signify. It simply looks for the pattern — it has no inkling of the concept behind the pattern.

But in semantic search, every bit of information is defined by potentially dozens of meaningful concepts. When a copywriter invoices for his or her work, for example, the date could be defined in terms of calendar, invoice, billing period, and so on. All these definitions for one piece of information are called ‘metadata’, or information about information.

Collections of agreed metadata terms for a particular field or task, like medicine or accounting, are called ontologies.

So the computer not only searches for the term, it searches for related metadata that defines types of information in specific ways. In reality, the computer still does not ‘understand’ a concept in its semantic search — it continues to look for patterns of letters. But because the concepts behind the search terms are included, it can return results based on concepts as well as text patterns.

Google improves movie searches

Vic Laurie — Thu, 12 Nov 2009 20:48:32 +0000

Updates and changes to search keep pouring out of Google. Here’s one for movie fans. Google has announced:

Did you know you can immediately discover movie times and locations by typing a simple search on Google? For example, search for [where the wild things are] and you’ll see times and places to watch the film, or try [glendale 18 los angeles] to see movies playing at your local theater. You can also click on the showtimes search result to get more complete movie listings and information.

Today, we’re making a few improvements to movie results on Google, including more detailed movie pages, genre filters and a new map view.

Bing updated

Vic Laurie — Wed, 11 Nov 2009 19:14:24 +0000

As posted yesterday, Google is updating its search engine. Not to be outdone, Microsoft isn’t sitting still and will be bringing out a new version of Bing. Mary Jo Foley has some details at her ZDNet blog.

New version of Google search coming soon?

Vic Laurie — Tue, 10 Nov 2009 18:03:29 +0000

As previously posted, Google has been working on an update to the architecture of its search engine. The project is called “Caffeine” and an experimental site was made available for trying out the new search engine. That site has now been pulled and this announcement posted:

Based on the success we’ve seen, we believe Caffeine is ready for a larger audience. Soon we will activate Caffeine more widely, beginning with one data center. This sandbox is no longer necessary and has been retired, but we appreciate the testing and positive input that webmasters and publishers have given.

So it looks like Caffeine will soon begin to be phased in. At CNET, Lance Whitney comments:

Caffeine won’t change the look or feel of Google’s popular search engine but will work under the hood to improve its performance, reportedly delivering faster, better, and more flexible results. Though Google continually tweaks its search engine, Caffeine represents the first major enhancement to its search indexing since 2006.

More Google search functions

Vic Laurie — Tue, 03 Nov 2009 16:56:28 +0000

Yet more additions to Google search have come out.

One is called “related links”. Google Operating System describes the feature:

Google Related Links is a new Google Labs service that lets you add a list of related web pages and searches to your site. Unlike the homonymous service released by Google in 2006, the new Related Links restricts the results to your site.

A second addition is page previews; it is described:

Google added a new option to the web search toolbelt: page previews. If you click on “Show options” and select “Page previews” after performing a search, Google will show a longer snippet and a thumbnail for each search result.

Bad search results

Vic Laurie — Mon, 02 Nov 2009 10:44:07 +0000

Back in prehistoric times BG (Before Google), the content of the Internet was indexed by human editors. Thus, a listing in one of the search facilities back then was the result of some person or persons making a judgment on the quality and relevance of a link. However, as the content of the Internet exploded, the human editors could not keep up. Then Google came along.

The great contribution of Google was to devise superior ways for computers to do the work of searching and indexing the Internet. Google’s search algorithms gave better results than similar efforts by others and made Google the leading choice for searching. Most of the search facilities like AltaVista that were prominent BG faded into obscurity. Yahoo and Microsoft tried hard but remained behind Google in market share.

Considering the staggering amount of information that has to be found and sifted, computer search algorithms have done a pretty good job. In fact, it is quite a feat to scan your search query, sort through an enormous mountain of data, and return an ordered list of search results in a second or so. Still, it’s computers that are doing the searching and indexing, and computers lack the finesse of human editing. Search results are a long way from perfect.

An entire industry has developed that is devoted to finding ways to increase a site’s ranking in Google and other searches. SEO (Search Engine Optimization) is big business. Basically, it’s a game of trying to get one up on the computer search algorithms used by Google and others. Malware writers and cybercriminals also try to game the algorithms. And, of course, advertisers try to get their products in the first rankings.

All of this means that search listings are a mix of relevant, useless, and sometimes even harmful links. It means that a lot of trash is given high ranking; it also means that a lot of good sites are buried way down the list and never see the light of day.

I haven’t made any quantitative study but my personal perception is that Google’s listings contain more trash than they used to. The amount of information that has to be searched is growing exponentially so I suppose the quality of search listings is fated to get worse. Whether Google, Yahoo, and Bing can do much about the deterioration, I don’t know. Is adding some human input to rankings practical? Yahoo recently got a patent for combining human and computer ranking. But the patent was applied for seven years ago and Yahoo has basically given up search in its deal with Microsoft.

I’m not the only one who sees search getting worse. Search authority Danny Sullivan has made several posts lately about the situation. Recently, he wrote, The Myth Of Great Search Engine Results, where he shows examples of poor search results for all three major services. He concludes:

To me, it feels like they’re getting worse, not better. But I can’t document that. What I can do is demonstrate without much difficulty, for areas where I have subject expertise, how bad they can be. They get by because along with the bad, there’s enough good. But they should be better than this.

Searching for music

Vic Laurie — Thu, 29 Oct 2009 10:09:08 +0000

It’s been hard to keep up with Google announcements this week. Here’s another one. Google is introducing a music search feature:

Every day we get millions of search queries about music. You want to know more about your favorite artists, find that new album or iconic song or figure out the name of that tune stuck in your head. In fact, according to Insights for Search, two of the top 10 queries in the U.S. are music-related. But often, if your answer is in a song, it can take a while to get there. We call this “time to result” — and we’re always looking for ways to reduce it.

Today, we’re rolling out a search feature that does just that by enabling you to search and more easily discover millions of songs, all via a simple Google web search. If you’re searching for music, “time to result” is really “time to music.” Now, when you enter a music-related query — like the name of a song, artist or album — your search results will include links to an audio preview of those songs provided by our music search partners MySpace (which just acquired iLike) or Lala. When you click the result you’ll be able to listen to an audio preview of the song directly from one of those partners. For example, if I search for [21st century breakdown], the first results provide links to songs from Green Day’s new album. MySpace and Lala also provide links to purchase the full song.

Google search adds similar images

Vic Laurie — Wed, 28 Oct 2009 15:06:01 +0000

Doing an image search for pictures? Find one you like and now you can search for similar images at Google image search. The Official Google Blog describes the new feature:

Today, we’re happy to announce that Similar Images is graduating from Google Labs and becoming a permanent feature in Google Images. You can try it out by clicking on “Find similar images” below the most popular images in our search results. For example, if you search for jaguar, you can use the “Find similar images” link to find more pictures of the car or the animal.

Is real time search a moneymaker?

Vic Laurie — Wed, 28 Oct 2009 10:35:47 +0000

Microsoft and Google have been falling all over themselves trying to one-up each other in searching Twitter and Facebook. Aside from the pleasure of elbowing a rival, is there any money to be made? Monday’s New York Times business section looked at this question. The gist is that no one is sure. Neither Microsoft nor Google claim that advertising money is going to roll in. They just want to attract users to their search facility:

Sean Suchter, a general manager for search technology at Microsoft, said that he expected real-time search would eventually become lucrative. But he added that for now, “The goal is definitely to drive user value.”

SIMILARLY, Google said that real-time search is valuable, though not necessarily because the queries will generate as much cash as regular searches.

“We don’t know enough about what kinds of queries people would issue against real-time data to know how monetizable it is,” said Marissa Mayer, Google’s vice president of search products and user experience.

Google wants the Twitter data primarily because its mission is to be comprehensive: Google wants to organize all of the world’s information, including the Web’s fleeting real-time conversations.

“We do know that comprehensiveness has a commercial benefit for our business,” she said. In other words, comprehensiveness keeps people searching on Google.

Social searching from Google

Vic Laurie — Tue, 27 Oct 2009 15:25:10 +0000

Google has unveiled another form of search. It’s social searching.

More easily find relevant blogs, reviews and other public content from your social circle.

Sign in to Google and do a search. If there’s relevant web content written by people in your social circle, it will automatically show up at the bottom of your search results under a section called “Results from people in your social circle.”

To see even more social content you can click the “Show options” link at the top of the results page and then click on the “Social” link.

What is your social circle? It’s a combination of your Gmail chat buddies, your Gmail contacts friends, family and co-worker groups, and people you’re publicly connected to on other social sites (such as Twitter and FriendFeed).

More about Google Social Search is at Search Engine Watch. Also, Danny Sullivan discusses the service at Search Engine Land.