1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Google indexing unwanted pages in wordpress

Discussion in 'BlackHat Lounge' started by psychosimo, Aug 30, 2013.

  1. psychosimo

    psychosimo Regular Member

    Joined:
    Oct 14, 2012
    Messages:
    415
    Likes Received:
    145
    Gender:
    Male
    Hi , i wanted to see what pages on my website google indexed , so i used the site:www.mysite.com thing and i've noticed that among the indexed pages there is some pages from the /wp-includes/ folder (yeah my website in under wordpress) , so i want to know is it normal that those pages are indexed or there is a problem with that ?? here is how my robots.txt looks :
    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
     
  2. WhitePassion

    WhitePassion Elite Member

    Joined:
    Feb 6, 2012
    Messages:
    1,874
    Likes Received:
    1,652
    Occupation:
    Marketing Wizard
    Location:
    Location Independent
    Google noindex wordpress plugin ;)
     
  3. EricEstate

    EricEstate Newbie

    Joined:
    Aug 28, 2013
    Messages:
    2
    Likes Received:
    2
    Occupation:
    Real Estate Agent
    Location:
    Austin TX
    Home Page:
    Make sure your server is set up right, too. If it's not a robots.txt file can be useless.
     
  4. psychosimo

    psychosimo Regular Member

    Joined:
    Oct 14, 2012
    Messages:
    415
    Likes Received:
    145
    Gender:
    Male
    can you give me a link to that plugin plz , i couldn't find it
     
  5. psychosimo

    psychosimo Regular Member

    Joined:
    Oct 14, 2012
    Messages:
    415
    Likes Received:
    145
    Gender:
    Male
    How ?? my website is hosted on a shared hosting plan (hostgator)
     
  6. metalfreek

    metalfreek Junior Member

    Joined:
    May 21, 2010
    Messages:
    139
    Likes Received:
    58
    I thought google by default would not index wp-admin folder and files.
     
  7. psychosimo

    psychosimo Regular Member

    Joined:
    Oct 14, 2012
    Messages:
    415
    Likes Received:
    145
    Gender:
    Male
    that's what i thought :( but they are being indexed
     
  8. psychosimo

    psychosimo Regular Member

    Joined:
    Oct 14, 2012
    Messages:
    415
    Likes Received:
    145
    Gender:
    Male
    so ?? any help here ?