1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Backlink Bots finding sites even with htaccess blocked

Discussion in 'Black Hat SEO' started by Openside, Apr 18, 2016.

  1. Openside

    Openside Power Member

    Joined:
    Dec 16, 2013
    Messages:
    543
    Likes Received:
    95
    Home Page:
    Hi guys,

    I have some of my sites blocking backlink checkers using htaccess but majestic is still showing some of them even though they all use the same htaccess code.

    Anyone much experience with this or any ideas?

    Thanks
     
  2. jacker818

    jacker818 Power Member

    Joined:
    Jun 10, 2010
    Messages:
    742
    Likes Received:
    245
    Gender:
    Male
    Occupation:
    Entrepreneur | SEO
    Location:
    Los Angeles
    Maybe it got picked up before you blocked the robots?
     
  3. starki

    starki Power Member

    Joined:
    Jul 17, 2012
    Messages:
    710
    Likes Received:
    233
    To block a bot, you can either block IPs or user agents. If the bot uses a different IP or user agent, it slips through. I don't know about MOZ, but everybody offering a paid tools wants to offer reliable data. I wouldn't be surprised if some of these tools crawl again "incognito", disguised as some user, if they discover that they are blocked.
     
    • Thanks Thanks x 1