meta- (prefix) - Beyond; transcending; more comprehensive. - At a higher state of development.A study by the NEC Research Institute says the Internet has exploded to more than 320 million Web pages. According to that study, even the most comprehensive search engine, HotBot, indexes only 34% of the web. By comparison, AltaVista was rated at 28%, Northern Light at 20%, Excite at 14%, and Lycos at 3%. Because many web sites are indexed by one search engine but not another, doing a search with multiple engines can reveal different sites.
The report offers good advice: finding what you want can be best achieved using a "meta-search engine." A meta-search engine is a tool that sends your query to several search engines at once.
Although they often provide more links, meta-search engines are a bit slower than regular search engines. Since a meta-search tool sends your request to several search engines and processes the results before delivering them to you, it must wait for all of them to reply before showing you the results.
As if picking a search engine wasn't hard enough, you have a choice of several meta-search engines, too. Here is a roundup of the best ones.
-.-.- MetaCrawler -.-.-
MetaCrawler (http://www.metacrawler.com) is a simple but effective meta-search engine. When you enter your search term, MetaCrawler starts parallel searches on AltaVista, Excite, Infoseek, Lycos, Webcrawler and Yahoo. It sorts and combines the results into a single list. Those results are ranked on a scale from zero to 1000 -- the most relevant hits have higher numbers and are listed first.
Its power search page (http://www.metacrawler.com/index_power.html) allows you to limit its searching by domain name: you can single out sites on a particular continent, or restrict results to U.S. educational, government or commercial sites. As with most of the meta-search engines we'll look at here, you can set the maximum time MetaCrawler will wait for results from the search engines. 10 seconds is the default. For a more thorough search, you can increase the time to an agonizing two minutes. Finally, you can adjust the number of hits MetaCrawler will request from each source. Ten is the default -- this will provide a few, highly-relevant hits.
MetaCrawler does a great job of separating the wheat from the chaff. If you're looking for a list of sites most relevant to a topic, quickly, it does an admirable job.
-.-.- DogPile -.-.-
DogPile (http://www.dogpile.com) does things a bit differently. It serves up the results from a bunch of search engines without sorting them or checking for duplicates. The results page shows the unedited hits from three search engines at a time. Pressing "Next Set of Search Engines" fetches the results from three different engines. DogPile is quite thorough -- it can perform searches with 13 web search engines, and (optionally) Usenet, FTP and newswires.
Sorting sites by the search engine that found them doesn't seem like the best way to go about it. The result is a lot of hits of variable relevance to your search topic. When you're looking for quantity, DogPile does the trick. On the other hand, if you're looking for just one or two of the best sites, DogPile can leave you barking up the wrong tree.
You can give DogPile a shorter leash using its "Custom Search" feature. With it, you can choose what search engines it will use, and in what order. Try this feature to make it query your two or three favorites in one fell swoop.
-.-.- MetaFind -.-.-
MetaFind (http://www.metafind.com) is another tool from the folks who created DogPile. I've got to admit that I like this version a whole lot more. MetaFind's saving grace is its sort option -- you can have it sort the results by keyword, alphabetically, by domain name or (if you insist) by search engine. Try the keyword sort, which separates results based on which of the words in your search phrase are most relevant. You can also opt to have MetaCrawler show or hide a short description of each site.
What tools does it use to search? Excite, AltaVista, Excite, InfoSeek, WebCrawler and PlanetSearch. MetaFind gets as many results as it can from each engine, so it usually returns a long list of hits. MetaFind does a good job of collecting and sorting data from other engines.
-.-.- SavvySearch -.-.-
Bookmark this one, because you'll never remember the URL (http://savvy.cs.colostate.edu:2000). SavvySearch is one of my favorite meta-search tools. It trolls 19 search engines, and allows you to choose brief, normal or verbose results output.
Pay attention to the "Integrate Results" button. When unchecked (the default), SavvySearch sorts sites by the search tool that found them. When selected, duplicate results are combined, and results are not separated by search engine. While this format is easier to interpret, you must wait until all search engines report (up to 45 seconds) before seeing any hits. SavvySearch's results are particularly easy to read and well-described.
At the time of writing, there is a more powerful search page being tested at http://guaraldi.cs.colostate.edu:2000/form?beta -- in addition to web sites, it can search for Usenet news, movies, software archives, technical reports and other information.
-.-.- All4One -.-.-
The All4One Search Machine (http://www.all4one.com) takes a unique slant on searching. When you enter a search term, All4One creates four frames in your browser window: each one contains the output from a different search engine. All4One doesn't try to interpret or sort the results -- it's just a quick way to do searches at AltaVista, WebCrawler, Lycos and Excite simultaneously. The four frames are rather small (especially if you have a 14" monitor). A little control panel at the top of the screen allows you to zoom in on any of the four windows for a closer look.
It's pretty simple but convenient. My only complaint about All4One is that you can't change the four search engines that it uses: if you'd rather it search HotBot (a fantastic search engine) instead of WebCrawler (a lousy one), you're out of luck.
-.-.- Super-Seek -.-.-
SuperSeek (http://www.super-seek.com) is similar to All4One: rather than putting each search engine in a frame, SuperSeek opens a browser window for each. You can choose which of eleven search engines to use. Pick your favorites carefully -- picking them all will mean 11 new browser windows to sift through.
This is another convenient tool for doing the same search on several search engines, but it, like All4One, doesn't provide the convenience of organizing or clustering the results.
=*=*= SITES MENTIONED IN THIS ISSUE =*=*==
All4One: http://www.all4one.com
DogPile http://www.dogpile.com
MetaCrawler: http://www.metacrawler.com
MetaFind: http://www.metafind.com
SavvySearch: http://savvy.cs.colostate.edu:2000
Super-Seek: http://www.super-seek.com