1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

<<how can i noindex wp pages bogging down my onpage seo?>>

Discussion in 'Black Hat SEO' started by entreprenuer, Jun 24, 2009.

  1. entreprenuer

    entreprenuer Regular Member

    Joined:
    Apr 19, 2009
    Messages:
    200
    Likes Received:
    73
    Location:
    Orlando Fl
    hey there,

    just as the title says I need to noindex a bunch of wordpress pages bogging down my seo.

    Pages like: "wp login" "error pages" and some weird pages I never even created are bogging down my onpage seo.

    What can I do to prevent google from indexing these pages?
     
  2. xhpdx

    xhpdx Regular Member

    Joined:
    Sep 21, 2008
    Messages:
    331
    Likes Received:
    2,160
    Occupation:
    Coder
    Location:
    EU
    dunno if this will help now, but you should use robots.txt with contents like

    Code:
    User-agent: *
    Disallow: /wp-admin
    Disallow: /another
    Disallow: /another2
    
    this will make all robots (search engines, bots who obey robots.txt) not visit the subfolders from the file

    more info about robots.txt standard
     
    • Thanks Thanks x 1
  3. entreprenuer

    entreprenuer Regular Member

    Joined:
    Apr 19, 2009
    Messages:
    200
    Likes Received:
    73
    Location:
    Orlando Fl
    ah man youre so awesome! i added some thanks and reputation.

    Now all I do is save the following text as a .txt file and place it in the .cgi-bin right?

    if not where do i place the robots.txt?
     
  4. xhpdx

    xhpdx Regular Member

    Joined:
    Sep 21, 2008
    Messages:
    331
    Likes Received:
    2,160
    Occupation:
    Coder
    Location:
    EU
    yes make it a txt file and place it in www or public_html folder, so it will be yourdomain.com/robots.txt not yourdomain/cgi-bin/robots.txt which won't work
     
    • Thanks Thanks x 1
  5. entreprenuer

    entreprenuer Regular Member

    Joined:
    Apr 19, 2009
    Messages:
    200
    Likes Received:
    73
    Location:
    Orlando Fl
    alright man. thanks again for the help.

    I just successfully uploaded the robots.txt file through my ftp client.

    check it out: hxxp://forexrobottrader.org/robots.txt
     
  6. xhpdx

    xhpdx Regular Member

    Joined:
    Sep 21, 2008
    Messages:
    331
    Likes Received:
    2,160
    Occupation:
    Coder
    Location:
    EU
    it looks good, but i am not quite sure whether a entry like

    Code:
    wp-login.php?redirect_to=
    
    will be obeyed, cause of parameters

    perhaps someone who knows more about robots.txt will say for sure
     
    • Thanks Thanks x 1
  7. adamster

    adamster Regular Member

    Joined:
    Nov 1, 2008
    Messages:
    209
    Likes Received:
    83
    NO index No follow wp plugin:
    Code:
    http://wordpress.org/extend/plugins/noindex-archives/
    This plugin works very well eventhough it wasn't updated since 2009-2-12. I recently upgraded about 40 sites using this plugin to wp 2.8 with no problem.
     
    • Thanks Thanks x 1