Ahrefs, majestic etc. spider list for .htaccess

Discussion in 'Black Hat SEO' started by Neocortx, Sep 12, 2014.

  1. Neocortx

    Neocortx Jr. VIP Jr. VIP

    Joined:
    Mar 2, 2013
    Messages:
    300
    Likes Received:
    186
    hey everybody,

    Some time ago I saw a thread where users shared a pretty big list for blocking spiders from most SEO bots in order to avoid competitors finding out about the PBN.

    Now that I need it, I just can't find it. I tried many different ways of searching, but nothing.

    Maybe someone has the link? Or any other place where I can find a big list.

    Thank you very much in advance.

    warm regards,
    neocortx
     
  2. Neocortx

    Neocortx Jr. VIP Jr. VIP

    Joined:
    Mar 2, 2013
    Messages:
    300
    Likes Received:
    186
  3. coitza

    coitza Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 26, 2007
    Messages:
    2,715
    Likes Received:
    722
    Occupation:
    freelancer
    Home Page:
    If you're using wordpress as a platform i think ithemes security has all this stuff in it, saw that it filled my htaccess with pretty much similar entries.
     
    • Thanks Thanks x 1
  4. poliofighterzone

    poliofighterzone Newbie

    Joined:
    May 23, 2014
    Messages:
    18
    Likes Received:
    1
    Spyderspanker is I think what the plugin is called for wordpress that does it.
     
  5. SEO Power

    SEO Power Elite Member

    Joined:
    Jul 14, 2014
    Messages:
    2,642
    Likes Received:
    683
    Occupation:
    Self employed
    Location:
    Houston, TX
    You are right. There is a pro version of the plugin if you want more features, but the free plugin should be enough.