Clues to Finding the Right Search Engine

Search engines like Yahoo, AltaVista, Lycos and Excite make it easy to find what you want on the Internet. But it’s also easy to be overwhelmed with too much information. And then there is the opposite problem: Sometimes you just can’t find what you’re looking for, even though you know it’s out there.

The key to finding information on the Internet is knowing which search engines to use and how to use them.

There are basically two types of search engines. Some, such as AltaVista (https://www.altavista.digital.com), Excite and Lycos employ software known as “spiders,” “robots” and “crawlers” that constantly search the Web and other parts of the Internet to create an index or catalog of sites based on the words they contain. AltaVista’s software, for example, looks at every word on every page as it travels the Internet 24 hours a day, sucking up information. No wonder it already contains about 30 million listings.

The other type of search engine, called a directory, is considerably more modest. Yahoo, which is the largest and most popular directory site, has about 500,000 listings divided into 25,000 categories. That’s still a lot of data, but the results of a Yahoo search tend to be a lot more concise than what you get from a spider-type search engine.

Some of the search engines are actually a combination of the two. Lycos, for example, has a crawler that automatically locates sites, but it also has Web guides that are organized by subject. It also offers a service called “top 5%” where reviewers hand-pick sites they think are worth looking at. If you do a Web search, the service looks first at its own structured database and then to the automated listings unearthed by its crawler.

Recently, Encyclopaedia Britannica launched EBlast.Com, a Web navigation service with 125,000 reviewed sites and an arrangement with AltaVista that allows you the option of including AltaVista searches along with the rated sites.

My search strategy depends on what I’m looking for. If I want to find a company’s Web site, I generally first try typing in the company name or abbreviation followed by .com. If guessing doesn’t work, my next stop is usually https://www.yahoo.com because its database, while extensive, generally yields manageable results.

If I don’t’ find it in Yahoo and it’s on the Web, I’ll probably find it in AltaVista, Lycos or Excite. But getting the most out of these sites requires a bit of finesse.

Let’s assume you are looking for the home page of United Airlines. If you typed United in AltaVista, you’d get more than 13 million sites. If you typed United Airlines, you’d get about the same amount, but the service is smart enough to put the United Airlines sites near the front because it assumes that the two words strung together are probably what you’re looking for. If you put United Airlines in quotes, you’d get the 36,000 sites in the database with the full name of the airline. That’s a lot less than 13 million but still too many to weed through.

But if you entered “United Airlines” in Yahoo, you’d find only 24 listings, beginning with the airline’s home page.

While Yahoo is clearly my search engine of choice for larger companies and organizations, it’s not likely to contain information on more obscure topics, which is the strong point of the crawler sites.

I often use sites like AltaVista, Excite and Lycos to find information about companies and people I’m writing about. I’ve discovered all sorts of tidbits about people in the PC industry, even if they aren’t famous.

But even when searching for relatively unknown organizations or individuals, you can sometimes be overwhelmed if you don’t limit the search. In addition to putting strings of text in quotes, you need to become familiar with the Web site’s tools for limiting or expanding your search.

Each site has slightly different procedures, which you can read about on their help pages. These techniques, which use “Boolean” logic, generally include modifier terms such as “and,” “or,” “not” and “near.” Instead of using “and,” some search engines, including AltaVista, have you place a + next to every word that must be in the document and a - next to any words you want to exclude. Typing +bill +clinton -Monica would give you documents that contained Bill and Clinton but would eliminate the nearly 19,000 Web documents that refer to both Clinton and Monica Lewinsky.

The “or” modifier can also be very useful. If you search for “Bill Clinton” in AltaVista, you’ll get about 142,000 documents. But if you search for “Bill Clinton” or “William Clinton” or “William Jefferson Clinton,” you’ll get about 150,000 matches. “Near” limits the search to words that are near each other in a document.

Excite offers the same Boolean features as AltaVista but makes life a lot easier by offering a power search page where you can enter terms on a form and select modifiers from a pull-down menu.

There are, of course, several other search engines. One of the most versatile, Hotbot.com, lets you use Boolean phrases but also has a pull-down menu that lets you limit your search by date, region or domain type and other criteria.

Northern Light (https://www.northernlight.com) offers the usual Web searching facilities, but it organizes the results into custom search folders that make it easier to find material on specific subjects.

AskJeeves (https://www.aj.com) is probably the Web’s most unusual search engine because it lets you type in questions in natural language. I used it a few weeks ago when, for a column, I needed to know the name of the inventor of the typewriter. It would have been incredibly difficult to find this information with other search engines. But when I asked, “Who invented the typewriter,” AskJeeves came back with Web pages with precisely the right answer.

*

Larry Magid can be e-mailed at [email protected]. His Web page is at https://www.larrysworld.com or keyword “LarryMagid” on AOL.