Blocking Ahrefs, MajesticSEO, OpenSiteExplorer and Other Backlink Bots

hnmfvnxe

Regular Member
Joined
Nov 10, 2012
Messages
280
Reaction score
82
I know using the htaccess and robots files to protect privately owned networks. But from what I understand they will continue to gather backlinks from other websites/sources you don't own (bookmarks, forum, web 2.0, wiki, articles, etc.)

Is there anyway to block these bots from gathering ALL your backlink data?
 
I don't see how it would be possible, no.

Edit: Unless maybe you bought a domain, 301 redirected it but blocked robots and then built links to that website? That's kinda out there, but I guess that could work?
 
I don't see how it would be possible, no.

Edit: Unless maybe you bought a domain, 301 redirected it but blocked robots and then built links to that website? That's kinda out there, but I guess that could work?

I was afraid of that. Thanks for the reply.
 
If you have modrewrite working on your server, this is the code that you need to put in .htaacess file

SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
SetEnvIfNoCase User-Agent .*sitebot.* bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
 
Here is a better thing, make a legal notice on your website, and once they scrape your website, SUE THEM ! :D
 
If you have modrewrite working on your server, this is the code that you need to put in .htaacess file

This will only work to protect websites you own like blog networks.

They will still gather your backlink data from bookmarks, forum, web 2.0, wiki, articles, etc.
 
I would think using this on your Tier 1 websites (wherever possible) would be a good idea. This way whenever your competitors use ahrefs or OSE or the like to check out your money site, your best links (tier 1 your PBN) won't show up at all. Seems like a pretty effective way to keep your competition from easily picking off/reporting your best links.
 
there is a plugin called spider spanker for your sites, not sure what you can do about others though. Probably not much.
 
Back
Top