User-agent: * # directed to all spiders Disallow: /cgi-bin/ Disallow: /cont/ Disallow: /css/ Disallow: /admin/ Disallow: /cache/ Disallow: /docs/ Disallow: /logs/