Robots.txt for PBN

Discussion in 'Black Hat SEO' started by rakeem, Feb 25, 2015.

  1. rakeem

    rakeem Junior Member

    Joined:
    Nov 2, 2011
    Messages:
    133
    Likes Received:
    47
    Hi dudes,

    i want no plugin for hiding my PBN. Please post the codes for robots.txt or .htaccess to secure PBN.

    I would appreciate it if you dont post any plugins for this.

    Thanks... Cheers.
     
  2. watsonovedades

    watsonovedades Senior Member

    Joined:
    Mar 10, 2011
    Messages:
    921
    Likes Received:
    235
    Location:
    Mexico
    you mean you want to hide your links and avoid the google spiders to crawl them?
     
  3. nycdude

    nycdude Power Member

    Joined:
    Oct 1, 2009
    Messages:
    515
    Likes Received:
    573
    Location:
    Mazatlán
    If you mean to keep spiders of your page it's on a robots.txt file with this:

    Code:
    User-agent:*
    
    Disallow: /
    Upload that to the path you want to keep the spiders away ... if that's what you want to do that it.
     
  4. jollyman

    jollyman Elite Member

    Joined:
    Aug 29, 2013
    Messages:
    1,814
    Likes Received:
    287
    Gender:
    Male
    Occupation:
    Digital marketing
    Location:
    Somewhere around BHW
  5. Lalalaenhund

    Lalalaenhund Power Member

    Joined:
    Sep 14, 2010
    Messages:
    746
    Likes Received:
    420
    Location:
    errywhere
    Correct me if I'm wrong but if this is for a PBN, I think you need to add a line to allow Google spiders, otherwise your network will be kind of useless.
     
  6. rakeem

    rakeem Junior Member

    Joined:
    Nov 2, 2011
    Messages:
    133
    Likes Received:
    47
    i want to block all the unnessasary bots to hide my PBN. I know i can use plungins like spider spanker pro, etc. but as i said i dont want any plugin.

    Cheers...
     
  7. Aty

    Aty Jr. VIP Jr. VIP

    Joined:
    Jan 27, 2011
    Messages:
    6,243
    Likes Received:
    4,259
    Home Page:
    • Thanks Thanks x 1
  8. accelerator_dd

    accelerator_dd Jr. VIP Jr. VIP

    Joined:
    May 14, 2010
    Messages:
    2,448
    Likes Received:
    1,010
    Occupation:
    SEO
    Location:
    IM Wonderland
    Or just:

    You are better off whitelisting a few than blacklisting a bunch.
     
    • Thanks Thanks x 2
  9. oscilay

    oscilay Regular Member

    Joined:
    Oct 30, 2013
    Messages:
    476
    Likes Received:
    406
    Don't do that for every website as that s a footprint in case of manual inspection from big G.
     
  10. rakeem

    rakeem Junior Member

    Joined:
    Nov 2, 2011
    Messages:
    133
    Likes Received:
    47
    i guess you are right :)
     
    • Thanks Thanks x 1
  11. rakeem

    rakeem Junior Member

    Joined:
    Nov 2, 2011
    Messages:
    133
    Likes Received:
    47
    that means not to hide the PBN. makes no sense.
     
  12. oscilay

    oscilay Regular Member

    Joined:
    Oct 30, 2013
    Messages:
    476
    Likes Received:
    406
    Ok. Don`t post here when your PBN will be deindexed.