User-agent: WebCrawler Disallow: / user-agent: * dasallow: /cgi-bin disallow: /strefa