# /robots.txt file for wmusearch

# /User-agent: wmusearch

User-agent: *
Disallow: /*.cgi 
Disallow: /*.cgi (The wmusearch will not be allowed to crawl any CGI files.) 
Disallow: /*.pl (The wmusearch will not be allowed to crawl any Perl scripts.) 
Allow: /$ (The wmusearch is allowed to crawl everything else.) 


# /User-agent: *
# /Disallow: /