// Used to disallow everything (prompted by the CPU spike). User-agent: * // Disallow: / Disallow: /blog/downloads/ Disallow: /wp-* // Disallow: /blog/images/ Disallow: /blog/wp-admin/ Disallow: /blog/wp-content/ Disallow: /blog/wp-images/ Disallow: /blog/wp-includes/ Disallow: /blog/Add-ins stored/ User-agent MJ12bot Disallow: / User-agent Psbot Disallow: / User-agent slow-crawler Disallow: / User-agent IRLbot Disallow: / User-agent twiceler Disallow: / User-agent sogou spider Disallow: / // Done