1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How can I stop google from indexing some of my pages?

Discussion in 'Blogging' started by patmiller27, Oct 18, 2010.

  1. patmiller27

    patmiller27 BANNED BANNED

    Joined:
    Jul 25, 2010
    Messages:
    179
    Likes Received:
    190
    Can anybody please tell me how can I stop Google from indexing some/all of my pages on my Wordpress blog. I still want it to index and show my posts... but not the pages.
     
  2. wickedguy

    wickedguy Supreme Member

    Joined:
    Jul 22, 2009
    Messages:
    1,402
    Likes Received:
    1,379
    Location:
    BHW--> South Africa
    Home Page:
    Add this in between the <head> </head> tags
     
    • Thanks Thanks x 1
    Last edited: Oct 18, 2010
  3. goldstrike4u

    goldstrike4u Junior Member

    Joined:
    Dec 22, 2007
    Messages:
    189
    Likes Received:
    124
    Or use the robots.txt file. Just do a google search for robots.txt and you will get easy instructions. Even simpler, as you are using wordpress use a plugin:
    http://yoast.com/wordpress/seo//
     
    • Thanks Thanks x 1
    Last edited: Oct 18, 2010
  4. patmiller27

    patmiller27 BANNED BANNED

    Joined:
    Jul 25, 2010
    Messages:
    179
    Likes Received:
    190
    I saw somewhere that this way Google will still index these pages... however, the would not be shown in the search results. I want it not to index it at all as they contain duplicate content.

    I know abt robots.txt but was a bit confused on the exact syntax that I need to use...suppose my page url is this...

    Code:
    http://www.xyz.com/abc/
    Then would it be fine to add a line as
    Code:
    Disallow: /abc
    Would this solve my purpose... or I am missing anything?
     
  5. detox1978

    detox1978 Junior Member

    Joined:
    Jan 16, 2009
    Messages:
    110
    Likes Received:
    33
    Bit of a novice mistake but a lot of websites put their download link into the robot.txt, a bit silly since it can be viewed by everyone.
     
  6. cool5now

    cool5now Registered Member

    Joined:
    Nov 12, 2009
    Messages:
    85
    Likes Received:
    11
    Robots.txt won't stop google indexing pages. I've had visits from google to pages banned by robots.txt

    It will stop them crawling the page but if there are external or internal links to the pages it'll still get indexed with the filename as a title and no description in Google serps.

    meta noindex is the way forwards as previously mentioned by wicked fire.... but this means google will have to crawl your page to see it. :rollseyes:
     
  7. jstratford

    jstratford Newbie

    Joined:
    Feb 22, 2009
    Messages:
    26
    Likes Received:
    2
    I'd use another approach. Utilize a duplicate content WordPress plugin that gives the appearance of unique content. That should help you overcome your concern, though I can't say I understand why you'd have duplicate content on your site. It's a good way to get penalized. :(
     
  8. thevil

    thevil Junior Member

    Joined:
    Aug 17, 2010
    Messages:
    190
    Likes Received:
    189
    If you want to block Google bots or anyone for that matter in accessing certain directories, modify your .htaccess file to include
    Code:
    deny from all
    
    robots.txt will disallow bots to show the index, but bots will still 'access' or index it, just not show it in SERPS. htaccess blocks them completely
     
    • Thanks Thanks x 1
  9. jackyman

    jackyman Junior Member

    Joined:
    Nov 22, 2010
    Messages:
    130
    Likes Received:
    6
    Home Page:
    Edit your robot.txt file with "disallow" of the page which you want to hide
     
  10. blacksmith71

    blacksmith71 Newbie

    Joined:
    Aug 10, 2008
    Messages:
    19
    Likes Received:
    6
    Occupation:
    Las Vegas Nightlife Concierge
    Location:
    Las Vegas, NV
    OK...

    Sorry for thee "noob" question...

    But why wouldn't you want google to index your pages, and index the posts instead???