CGI and Perl

Search the Web with WWW::Search

For searching outside of your site, a set of modules called WWW::Search has been written by John Heidemann. WWW::Search is a collection of Perl modules which provides a common API to most popular WWW search engines. As of this writing, there are modules which support AltaVista, Yahoo!, Lycos, Hotbot, and WebCrawler. The author is currently developing more modules for other search engines and more sophisticated clients and examples. The latest version of the WWW::Search module can be found at: http://www.isi.edu/lsam/tools/WWW_SEARCH/

Installation of the module requires Perl5 version 5.003, and is very straightforward. Using the module to generate custom queries to a search engine is very simple.

First, the type of search engine must be defined. Check the documentation to see if the search engine you wish to query is supported, then create a new search:

$search = new WWW::Search(`SearchEngineName');

An example would be:

$search = new WWW::Search(`AltaVista');

Then specify the query string. This string is made up of some specific name value pairs, and is URI encoded.

$search->native_query(`search-engine-specific+query+string');

Here's a documented example which performs an AltaVista search, then prints the URIs resulting from the search. Note that you could easily add nice custom formatting of the results in the while loop.

my($search) = new WWW::Search::AltaVista;
 $search->native_query(WWW::Search::escape_query($query));
 my($result);
 while ($result = $search->next_result()) {
 print $result->url, "\n";
 };

Listing 10.1 is code from search.PL, a small example included with the WWW::Search distribution. This example illustrates the usage of the Search Library. Figure 10.2 depicts an example of output that could easily be generated by search.PL.