This chapter explains how to use SWISH and WWWWAIS, Stronghold's standalone indexing and search facilities, including how to

• configure SWISH and WWWWAIS
  
  
• create a site index
  
  
• create an HTML search interface
  
  

Stronghold comes with two standalone programs for indexing your site and offering search capabilities to users:

• SWISH, the Simple Web Indexing System for Humans. You can run SWISH periodically to search the files in your Web tree, then save the results in a complete index file.
  
  
• WWWWAIS, a CGI program that searches the SWISH index file to generate responses to user search queries. When WWWWAIS receives a user search request, it scans the index file for entries that match the search parameters.
  
  

Stronghold's installation program places SWISH in ServerRoot/swish, and its configuration file is ServerRoot/conf/swish.conf. Like httpd.conf, it's a simple text file, but it does not use wrappers. The default configuration file is included in Appendix D.

This section explains the SWISH configuration directives, including ReplaceRules and FileRules, the directives that set the indexing parameters.

• Syntax: IndexDir directory
  
• Context: swish.conf
  

  IndexDir sets the directory that SWISH indexes. Usually, this is the same as the value you set for the DocumentRoot directive in httpd.conf. You can use more than one of these directives, although each can only take one value. For example, if the Web documents for your virtual hosts are in different directories than the main server documents, you can enter something like this:

  IndexDir /usr/local/www/htdocs
  IndexDir /usr/local/www/vhosts/vhost1
  IndexDir /usr/local/www/vhosts/vhost2
  IndexDir /usr/local/www/vhosts/vhost2
  ...
  
  

  SWISH indexes each directory recursively.

• Syntax: IndexFile filename.swish
  
• Context: swish.conf
  

  This directive sets the path to the index file, where SWISH saves the results of each indexing sweep. Be sure to include a .swish filename suffix.

• Syntax: IndexOnly .suffix1 [.suffix2 .suffix3 ...]
  
• Context: swish.conf
  

  IndexOnly specifies the types of file SWISH is allowed to index, using their filename suffixes. For example, you can limit the index to HTML and PHP files like this:

  IndexOnly .html .phtml .php
  
  

  This directive is case-insensitive. If you omit it, SWISH indexes every file in the directories specified by IndexDir.

• Syntax: IndexReport 0|1|2|3
  
• Context: swish.conf
  

  This directive sets the reporting option, which can be an integer from 0 through 3, 3 being the most verbose output option.

• Syntax: FollowSymLinks yes|no
  
• Context: swish.conf
  

  When FollowSymLinks is set to "yes," SWISH follows symbolic links while indexing. When set to "no," SWISH ignores them.

• Syntax: NoContents .suffix1 [.suffix2 .suffix3 ...]
  
• Context: swish.conf
  

  SWISH can index entire files or only their filenames. NoContents sets the suffixes of files whose contents SWISH should ignore. For example:

  NoContents .ps .gif .au .hqx .xbm .mpg .mpeg .pict .jpg .jpeg ...
  
  

  SWISH indexes only their filenames instead. This directive is case-insensitive.

• Syntax: IgnoreWords word1 [word2 word3 ...]
  
• Context: swish.conf
  

  Certain frequently-occurring words are irrelevant for indexing and searching purposes, such as prepositions, pronouns, articles, and indexicals. IgnoreWords sets the list of words that SWISH ignores. SWISH comes with a default list of several hundred common words, and it adds this list if one of the values for IgnoreWords is "SwishDefault."

  Along with IgnoreLimit, this directive can save CPU resources and control the size of the index file.

• Syntax: IgnoreLimit percent integer
  
• Context: swish.conf
  

  IgnoreLimit, like IgnoreWords, provides a method of filtering out frequently-occurring words when indexing. Each IgnoreLimit directive takes two values:

  • A percentage of the total number of Web files. SWISH ignores words that occur in at least this percentage of files.
    
    
  • A number of files. SWISH ignores words that occur in at least this many Web files.
    
    

  For example:

  IgnoreLimit 80 256
  IgnoreLimit 50 50
  
  

  The first instance instructs SWISH to ignore words that occur in at least 80 percent of files and in at least 256 separate files. The second instance instructs SWISH to ignore words that occur in at least 50 percent of files and in at least 50 separate files.

• Syntax: IndexName "name"
  
• Context: swish.conf
  

  IndexName sets the title of the index file.

• Syntax: IndexDescription "description" |URL
  
• Context: swish.conf
  

  IndexDescription is a short description of the index file, or the URL of a description file.

• Syntax: IndexPointer URL
  
• Context: swish.conf
  

  URL is the location of your site's home page.

• Syntax: IndexAdmin "administrator information"
  
• Context: swish.conf
  

  IndexAdmin gives descriptive information about the administrator responsible for Web indexing. SWISH includes this information in the index file.

• Syntax: ReplaceRules replace|append|prepend "string" ["replace-string"]
  
• Context: swish.conf
  

  Since SWISH does not read the httpd.conf file, it knows nothing about the aliases and virtual hosts you've set up for your site. ReplaceRules operates on the paths of the files that SWISH indexes, converting them into URLs. For example:

  prepend "http://"
  replace "/usr/local/httpd/htdocs/" "www.mainhost.com/"
  replace "/usr/local/httpd/vhosts/vhost1/" "www.vhost1.com/"
  
  

• Syntax: FileRules operator string1 [string2 string3 ...]
  
• Context: swish.conf
  

  FileRules is the opposite of IndexOnly: instead of limiting the indexed files inclusively, it limits them exclusively. SWISH ignores all files that match the parameters you set with this directive. Operator is one of the following:

  Operator	Description
  pathname contains	This operator takes one or more strings, and SWISH ignores any directory or file whose path contains one of these strings.
  filename is	This operator is case-insensitive and specifies a single, exact filename, without a path.
  filename contains	This operator is case-sensitive and specifies one or more filename strings. SWISH ignores any file whose filename (not path) contains one of these strings.
  title contains	This operator is case-insensitive and specifies one or more strings. SWISH reads the contents of each <TITLE> tag in HTML, and ignores any file whose title contains one of these strings.
  directory contains	This operator is case-insensitive, and specifies one or more filename strings. SWISH ignores an entire directory if it contains a filename that includes one of these strings.

Stronghold's installation program places WWWWAIS in ServerRoot/cgi-bin, and its configuration file is ServerRoot/conf/wwwwais.conf. Like swish.conf, it's a simple text file that does not use containers. The default configuration file is included in Appendix E.

This section explains the WWWWAIS configuration directives.

• Syntax: PageTitle "title"|filename
  
• Context: wwwwais.conf
  

  This sets the title of the search results file. If the quoted value is a string, only the string is used. If the value is a filename, WWWWAIS prepends the contents of the file to the search results.

• Syntax: SelfURL "URL"
  
• Context: wwwwais.conf
  

  This is the URL of the WWWWAIS search engine.

• Syntax: MaxHits n
  
• Context: wwwwais.conf
  

  The integer value for MaxHits is the maximum number of search results WWWWAIS is allowed to return.

• Syntax: SortType type
  
• Context: wwwwais.conf
  

  WWWWAIS sorts its search results according to the value for SortType. Type can be one of the following:

  Type	Description
  score	Sort the results according to the relevancy scores assigned to each and list the most relevant results first.
  lines	Sort the results according to the lengths of the resulting documents, measured by the number of lines in each.
  bytes	Sort the results according to the byte size of each file found.
  title	Sort the results alphabetically by title.
  type	Sort the results by filetype.

• Syntax: AddrMask all|IP1 [IP2 ...]
  
• Context: wwwwais.conf
  

  AddrMask specifies the IP addresses that are authorized to access the search gateway. The value can be "all," in which case any host can use the gateway, or a list of IP numbers. IP values can include wildcard strings.

• Syntax: SwishBin path
  
• Context: wwwwais.conf
  

  SwishBin sets the path to the SWISH indexing engine.

• Syntax: SwishSource path "description"
  
• Context: wwwwais.conf
  

  This directive sets the path and description of the SWISH index file.

• Syntax: SourceRules replace|append|prepend arg1 [arg2 ...]
  
• Context: wwwwais.conf
  

  Search results are pathnames from the index file generated by SWISH. In order to make this information useful to users, you can use SourceRules to modify the paths. For example:

  SourceRules replace "/www/" "http://your.host.com/"
  
  

  This converts document root paths to proper URLs that take advantage of the DocumentRoot alias.

• Syntax: WaisSource path "description"
  
• Context: wwwwais.conf
  

  This directive sets the source description for WAIS sources. For WAISSEARCH sources, the syntax is

  WaisSource hostname port path "description"
  

• Syntax: UseIcons yes|no
  
• Context: wwwwais.conf
  

  WWWWAIS can include icons in the search results page, according to the option set in UseIcons.

• Syntax: IconUrl URL
  
• Context: wwwwais.conf
  

  This directive sets the location of your icons directory. WWWWAIS uses the icons in this location if UseIcons is set to "yes."

• Syntax: TypeDef .suffix "type description" iconfile MIME-type
  
• Context: wwwwais.conf
  

  TypeDef matches filename suffixes to filetype descriptions, icon files, and MIME types. WWWWAIS uses this information to generate and sort search response pages. Use as many of these directives as you need.

Each site on your server platform requires its own site index. For example, if you have many virtual hosts, users who accesses one host must be able to search that host without receiving results from other hosts. Before you can use the WWWWAIS search facility, you must use SWISH to create a site index for each virtual host on your server. You must also update these indexes periodically to ensure that search results are current. With a little creativity, you can create scripts that automate these tasks for you.

The SWISH executable is ServerRoot/swish/swish. To create a site index, run SWISH from the command line, using the following flags to specify options:

Flag	Description
-i dir|file	Create an index of the specified directory or file. Use this flag to index the directory that contains an individual virtual host's files. SWISH indexes directories recursively. You can specify multiple directories or files, separated by spaces.
-w word1 [word2 . . . ]	Search only for the specified keywords. You can use "and," "or," "not," and parentheses. The search is case-insensitive, and SWISH evaluates your criteria from left to right.
-t H|B|t|h|e|c	Search only the specified HTML tags: H = <HEAD> B = <BODY> t = <TITLE> h = <Hn> e = <EM> c = <!-- comment --!>
-f filename1 [filename2]	Create or search the specified file. If you are indexing, you can specify only one file-the index file for SWISH to create. The default is index.swish in the current directory. If you are searching, you can give a list of files to search.
-c filename	Index using the specified configuration file. You can use this flag to implement custom SWISH configurations for individual hosts, or to eliminate the need for command-line flags by including all options in the configuration file.
-v 0|1|2|3	Run with the specified level of verbosity. 0 is silence, and 3 is the most verbose. The default is 3.
-l	Follow symbolic links when indexing.
-m n	Return no more than n, the maximum number of results. When n is not specified, SWISH assumes 40. To set no limit on the number of results, specify 0 or "all," or omit this flag. The value given in the configuration file overrides any value you specify with this flag.
-M index1 index2 . . . filename	Merge the specified index files into one file, specified by filename. SWISH removes all redundant data during this operation.
-D filename	Decode the specified index file.
-V	Print the current version number.

Once you have an index file for a host, any HTML search interface for that host must reference the appropriate index file. Make sure the administrator of each virtual host has access to that host's index file, and to the instructions contained in the next section, "Creating a Search Interface."

WWWWAIS searches the site indexes created by SWISH. It supports the Boolean operators AND and OR, and can use either GET or POST. The basic form that passes search parameters to WWWWAIS looks like this:

<FORM METHOD=GET ACTION="/cgi-bin/wwwwais?sourcedir=/usr/local/www/htdocs/vhost/swish&source=index.swish"
Search for:
<INPUT TYPE=TEXT NAME="keywords" SIZE=40>
<INPUT TYPE=SUBMIT VALUE="Search">
</FORM>

You can customize any search form by appending search options to the URL in the ACTION field. You can use as many options as you like in a single form. Options strings must be separated from the ACTION path by a question mark (?). Options must be separated from each other by ampersands (&). For example:

ACTION=/cgi-bin/wwwwais?source=index.src&keywords=sample+search

The rest of this section lists the available search options. Many of them are equivalent to certain configuration directives and can be used to override the settings in the configuration file. If you have multiple virtual hosts, it's important to specify the source index file in each search form using the sourcedir and source options. If you have only one host, you can simply set the source index in the configuration file.

• Syntax: sourcedir=path
  
• Context: ACTION
  

  This option specifies the directory that contains the index database. This setting is especially useful if you have several databases in the same directory, in which case WWWWAIS searches them all.

• Syntax: source=filename
  
• Context: ACTION
  

  The source option specifies which index database the search engine should search.

• Syntax: maxhits=n
  
• Context: ACTION
  

  This option sets the maximum number of URLs WWWWAIS returns to the user.

• Syntax: keywords=keyword1[+keyword2+keyword3 ...]
  
• Context: ACTION
  

  The keywords option specifies the keywords to search for.

• Syntax: isindex=keyword1[+keyword2+keyword3 ...]
  
• Context: ACTION
  

  The isindex option works identically to the keywords option.

• Syntax: sorttype=score|lines|bytes|title|type
  
• Context: ACTION
  

  This option sets the criteria that WWWWAIS uses to sort the results of a search.

• Syntax: version=true|false
  
• Context: ACTION
  

  When this option is set, WWWWAIS returns its version information and that of WAISQ or WAISSEARCH.

• Syntax: host=hostname
  
• Context: ACTION
  

  If you are using WAISSEARCH as your search mechanism, you can use this option to specify a remote host to search instead of your local host.

• Syntax: port=n
  
• Context: ACTION
  

  If you are using WAISSEARCH and the host option, you can use this option to specify the port number to access on the remote host you are searching.

• Syntax: useicons=yes|no
  
• Context: ACTION
  

  When the useicons option is set to "yes," WWWWAIS displays icons with the different files in the search results.

• Syntax: iconurl=URL
  
• Context: ACTION
  

  If you use the useicons option, use iconurl to specify the location of your icon files.

• Syntax: URL?selection="source+description"
  
• Context: ACTION
  

  This option specifies a source description, which must be one of the values set for WaisSource or SwishSource in the wwwwais.conf file.

• Syntax: URL?searchprog=waisq|waissearch|swish
  
• Context: ACTION
  

  You can use searchprog to specify one of these alternative search programs:

  • waisq
    
    
  • waissearch
    
    
  • swish
    
    

  Only SWISH comes with Stronghold. If you want to use WAISQ or WAISSEARCH, you must install them separately.

[Top] [Prev] [Next] [Last]

© 1998 C2Net International Feedback: stronghold-docs@c2.net