wp-include and wp-content

mondmond88 · Nov 6, 2009

normally for any site, we do use robots.txt to block spiders.
do i have to disallow my wp-content and wp-include folder?

Disallow: /wp-content/
Disallow: /wp-include/

and can we do it like this?
Disallow: /subdomain/wp-content/

or do we have to add a seperate robots.txt file for each subdomain?

icepay · Nov 6, 2009

If you are having latest wordpress 2.8 + then there is already a blank index.php, so if you type yourdomain.com/wp-content then it will show blank page.
No need to use robots.txt
I dont know about the old version. But still you can insert a blank index.php in to the folder if its is already not there.
Hope it helped

mondmond88 · Nov 6, 2009

I've included a blank html page
was just wondering if do i need to add disallow in the robots
im running 2.7.1

halfcut · Nov 6, 2009

This is what I have in mine. I'm running 2.8.5

Sitemap: /sitemap.xml

User-agent: *
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/themes
Disallow: wp-app.php
Disallow: wp-atom.php
Disallow: wp-blog-header.php
Disallow: wp-comments-post.php
Disallow: wp-config-sample.php
Disallow: wp-config.php
Disallow: wp-cron.php
Disallow: wp-feed.php
Disallow: wp-login.php
Disallow: wp-links-opml.php
Disallow: wp-mail.php
Disallow: wp-pass.php
Disallow: wp-rdf.php
Disallow: wp-register.php
Disallow: wp-rss.php
Disallow: wp-rss2.php
Disallow: wp-settings.php
Disallow: wp-trackback.php
Disallow: xmlrpc.php

...seems to work OK

wp-include and wp-content

mondmond88

BANNED

icepay

Junior Member

mondmond88

BANNED

halfcut

Newbie

Main Menu

Marketplace

Making Money

BlackHatWorld

Other