[ Help ] How to block Backlinks Crawlers

Vivekji424 · Oct 1, 2020

Hi,
I want to know how I can block backlinks Crawler such as Ahref, Moz, SEMrush etc. I have tried a couple of methods which alredy shared here, But Didn't Work for me if anyone can help me it would be great.

Thanks.

hideath · Oct 1, 2020

https://www.quora.com/How-can-you-b...EMRush-from-crawling-or-indexing-your-websitehere you go

if these bots already checked your website, I think you can't revert back the metrics
that will block them from any future crawls

SERPhustler · Oct 1, 2020

HTACCESS CODE

Code:

RewriteEngine On
RewriteBase /

SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
SetEnvIfNoCase User-Agent .*sitebot.* bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>

ROBOTS.TXT FILE CODE
robots.txt file code ( in robots.txt which should block everything except Google, Yahoo and Bing.)

Code:

User-agent: Googlebot
Disallow:

User-agent: msnbot
Disallow:

User-agent: Slurp
Disallow:

User-agent: Teoma
Disallow:

User-agent: rogerbot
Disallow: /

User-agent: exabot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: dotbot
Disallow: /

User-agent: gigabot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: *
Disallow: /

[ Help ] How to block Backlinks Crawlers

Vivekji424

Regular Member

hideath

Elite Member

SERPhustler

Elite Member

Main Menu

Marketplace

Making Money

BlackHat World