How to block ahrefs and other link checker bots to access my site

InstaRank

Elite Member

Apr 13, 2010

2,345

899

• Jan 28, 2013

• #1

I am looking for a step by step guide on how to block link checker networks like ahrefs bots to not visit my site , i tried doing it using robots.txt and it does not work, so i want to block them from htaccess,


thanks for any help.

 

N

nichelinks

Junior Member

Jan 11, 2013

117

37

• Jan 28, 2013

• #2

Upload robots txt file to your website using FTP, then your site wont be crawled by any bots. For more details just go through robotstxt org

 

A

auw21

Regular Member

Sep 18, 2010

285

79

• Feb 15, 2013

• #3

anyone can give a better advice?

 

Killix

BANNED

Nov 30, 2012

511

153

• Feb 15, 2013

• #4

I wasn't sure this was actually possible seen as links are on other sites pointing to you? Unless I misunderstood what you ment

 

P

proweb

Regular Member

Feb 24, 2011

291

147

• Feb 15, 2013

• #5

If you're trying to prevent people from seeing your backlinks you will need every site linking to you to block the Ahrefs robot.

Blocking Ahrefs from your site will only prevent Ahrefs from discovering the webpages your website is linking to.

 

X

xrfanatic

BANNED

Aug 28, 2010

466

201

• Feb 15, 2013

• #6

I think this is what you are looking for leena:

https://ahrefs.com/robot/index.php

Click to expand...

 

1

1+1=2

Junior Member

Jul 14, 2009

198

29

• Feb 15, 2013

• #7

blocking bots on your site doesnt stop it from collecting backlink profile of your site. They found your backlinks on the sites which have link point to you. If you need to stop Ahrefs or Majestic SEO from spying on your backlinks, you need to block them on every websites that you build backlinks on. THis is only possible if you own a private blog network

 

D

Dobermann35

Newbie

Oct 20, 2017

5

0

• Jul 12, 2018

• #8

1+1=2 said:

blocking bots on your site doesnt stop it from collecting backlink profile of your site. They found your backlinks on the sites which have link point to you. If you need to stop Ahrefs or Majestic SEO from spying on your backlinks, you need to block them on every websites that you build backlinks on. THis is only possible if you own a private blog network

Click to expand...

Are you for real? Ahrefs and all other bots can be blocked in Robots.txt. Put it in root directory of your site with User-agent: AhrefsBot
Disallow: /

I just talked to ahrefs support as well as semrush majetic etc.
They obey the robot.txt
Should your site info in ahrefs contain all data within 2 weeks you might consider doing the botblock through htaccess but I strongly encourage you to avoid second step without proper knowledge.

 

Start Earning

Senior Member

Apr 18, 2018

961

458

• Jul 12, 2018

• #9

I don't understand what that whole fuss about blocking PBN backlinks for Ahref and Majestic is about, given that:

1) Nobody is going to be interested in your backlink profile except competitive SEO practitioners (and Google)
2) Those guys (like us) can't be fooled anyway, because you find backlinks that the tools don't show but that are indexed by google easily on Google itself.
Yes, it's an additional manual step, but if you're that deep into analysing competition, a small step for you.

Maybe someone more advanced can shine some light on it, why it's still done?