User-agent: * Disallow: *.js Disallow: *.css Disallow: /archive/ Disallow: /assets/ Disallow: /cgi-bin/ Dissallow: /docs/