Google indexing unwanted pages in wordpress

Discussion in 'BlackHat Lounge' started by psychosimo, Aug 30, 2013.

  1. psychosimo

    psychosimo Jr. VIP Jr. VIP

    Joined:
    Oct 14, 2012
    Messages:
    423
    Likes Received:
    146
    Gender:
    Male
    Hi , i wanted to see what pages on my website google indexed , so i used the site:www.mysite.com thing and i've noticed that among the indexed pages there is some pages from the /wp-includes/ folder (yeah my website in under wordpress) , so i want to know is it normal that those pages are indexed or there is a problem with that ?? here is how my robots.txt looks :
    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /wp-admin/
    Disallow: /wp-includes/
     
  2. WhitePassion

    WhitePassion Elite Member

    Joined:
    Feb 6, 2012
    Messages:
    1,880
    Likes Received:
    1,661
    Occupation:
    Marketing Wizard
    Location:
    Location Independent
    Google noindex wordpress plugin ;)
     
  3. EricEstate

    EricEstate Newbie

    Joined:
    Aug 28, 2013
    Messages:
    2
    Likes Received:
    2
    Occupation:
    Real Estate Agent
    Location:
    Austin TX
    Make sure your server is set up right, too. If it's not a robots.txt file can be useless.
     
  4. psychosimo

    psychosimo Jr. VIP Jr. VIP

    Joined:
    Oct 14, 2012
    Messages:
    423
    Likes Received:
    146
    Gender:
    Male
    can you give me a link to that plugin plz , i couldn't find it
     
  5. psychosimo

    psychosimo Jr. VIP Jr. VIP

    Joined:
    Oct 14, 2012
    Messages:
    423
    Likes Received:
    146
    Gender:
    Male
    How ?? my website is hosted on a shared hosting plan (hostgator)
     
  6. metalfreek

    metalfreek Junior Member

    Joined:
    May 21, 2010
    Messages:
    139
    Likes Received:
    58
    I thought google by default would not index wp-admin folder and files.
     
  7. psychosimo

    psychosimo Jr. VIP Jr. VIP

    Joined:
    Oct 14, 2012
    Messages:
    423
    Likes Received:
    146
    Gender:
    Male
    that's what i thought :( but they are being indexed
     
  8. psychosimo

    psychosimo Jr. VIP Jr. VIP

    Joined:
    Oct 14, 2012
    Messages:
    423
    Likes Received:
    146
    Gender:
    Male
    so ?? any help here ?