TannerRitchie Web Applications

Advanced Web Application Development in the GTHA

Month: April 2015

  • Problem with default WordPress robots.txt

    Problem with default WordPress robots.txt

    A WordPress site we administer had the following automatically generated robots.txt file (in a fairly vanilla installation). User-agent: * Disallow: Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-login.php Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /trackback Disallow: /comments Note the lines which block indexing of theme and plugins. This used to be fine, but with Google now…