1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

wp-include and wp-content

Discussion in 'Blogging' started by mondmond88, Nov 6, 2009.

  1. mondmond88

    mondmond88 BANNED BANNED

    Joined:
    Apr 12, 2009
    Messages:
    200
    Likes Received:
    136
    normally for any site, we do use robots.txt to block spiders.
    do i have to disallow my wp-content and wp-include folder?

    Disallow: /wp-content/
    Disallow: /wp-include/

    and can we do it like this?
    Disallow: /subdomain/wp-content/

    or do we have to add a seperate robots.txt file for each subdomain?
     
    Last edited: Nov 6, 2009
  2. icepay

    icepay Junior Member

    Joined:
    Jan 13, 2009
    Messages:
    118
    Likes Received:
    19
    If you are having latest wordpress 2.8 + then there is already a blank index.php, so if you type yourdomain.com/wp-content then it will show blank page.
    No need to use robots.txt
    I dont know about the old version. But still you can insert a blank index.php in to the folder if its is already not there.
    Hope it helped
     
  3. mondmond88

    mondmond88 BANNED BANNED

    Joined:
    Apr 12, 2009
    Messages:
    200
    Likes Received:
    136
    I've included a blank html page
    was just wondering if do i need to add disallow in the robots
    im running 2.7.1
     
  4. halfcut

    halfcut Newbie

    Joined:
    Oct 30, 2009
    Messages:
    18
    Likes Received:
    9
    This is what I have in mine. I'm running 2.8.5

    ...seems to work OK
     
    • Thanks Thanks x 1