Should I block ahrefs crawler from my site?

[Fussballgott]

Do you do this with your money site?

 

[GeeLark AI]

if you feel insecure your competitors spying you, you can do this.

there is no problem if you use it, buy you can not see your website data via ahrefs.

 

[Mr Positive]

if it’s a pbn yes if not don’t do it no harm will come that’s what I feel

 

[IamKing]

Most of the PBN site owners block these tracking tools. It's not necessary to block your money site, ahrefs won't show all backlinks data.

 

[Rankster]

Most of the leading blogs, websites, service providers do not block backlink research sites like Ahrefs from crawling their sites. It helps you and your competitors to analyze each other backlinks. There is nothing wrong in this. This make the competition healthy.
Rather, if you are running a huge business and there have to maintain their backlink profile privacy. Then you've to block their sites from crawling.

 

[fb3003]

Do you do this with your money site?

i think if you want to take this approach, you need to block all BOTS and only accept google bot in your .htaccess file; otherwise, you will not be able to fully block ahref, majestic, semrush, etc. bots

still it does not make sense to do this on your money site, but on your pbns. i don't if you want to know.

 

[Fussballgott]

hmm okay thank you all for the answers. The reason I'm asking is because one of my competitors is doing this:
User-agent: Googlebot
Disallow:
User-agent: googlebot-image
Disallow: /
User-agent: googlebot-mobile
Disallow:
User-agent: MSNBot
Disallow:
User-agent: Slurp
Disallow:
User-agent: Teoma
Disallow: /
User-agent: Gigabot
Disallow: /
User-agent: Robozilla
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: baiduspider
Disallow: /
User-agent: naverbot
Disallow: /
User-agent: yeti
Disallow: /
User-agent: yahoo-mmcrawler
Disallow: /
User-agent: psbot
Disallow: /
User-agent: yahoo-blogs/v3.9
Disallow: /
User-agent: *
Disallow:
Can someone explain what he's doing there?

 

[dragonlube]

For SEOs that adopt black hat tactics like PBN those crawlers like majestic, ahrefs are a nightmare and can expose the network to competitors if left open, which in most cases will lead to a spam report causing the whole network to be de-indexed + a manual action applied to the money site if not a total deindexation.

 

[dragonlube]

He is using robots.txt method to block the crawlers.
This is used assuming that the companies which operate those crawlers do no use third party crawling services, that come under different user agents.
You can add few lines to robots.txt that can disallow most popular link crawlers.

 

[Fussballgott]

He is using robots.txt method to block the crawlers.
This is used assuming that the companies which operate those crawlers do no use third party crawling services, that come under different user agents.
You can add few lines to robots.txt that can disallow most popular link crawlers.

I understand that, but why is he blocking these specific crawlers? they don't seem like any SEO tools so whats the use in that?