Help with Blocking Ahrefs, Majestic, etc? (PBN)

[Joethemonkey101]

Hi, I'm working on building a PBN and I'm trying to block ahrefs and all the other backlink websites. I found some threads dealing with this but they are a couple of years old so I wanted to make sure everything is still the same.

Firstly, I'm editing the htaccess code of my PBN sites, but how does this block ahrefs? Does it mean that people can't look at the backlinks pointed at my PBN site or the PBN backlink pointed at my money site?

Secondly, is my htaccess file OK?

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>


SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
SetEnvIfNoCase User-Agent .*sitebot.* bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>


# END WordPress

And lastly, how can I edit my robots.txt file?

Thanks in advanced

 

[deancow]

htaccess looks fine, I wouldn't bother with a robots.txt file, if you block the same bots on every robots.txt it's a footprint and they might just ignore them anyway.

 

[Joethemonkey101]

htaccess looks fine, I wouldn't bother with a robots.txt file, if you block the same bots on every robots.txt it's a footprint and they might just ignore them anyway.

Is there a way to check if it's working?

 

[bahus]

If you're on Wordpress you can use KB Robots.txt plugin to edit your robots.txt file from within Wordpress.

 

[Joethemonkey101]

If you're on Wordpress you can use KB Robots.txt plugin to edit your robots.txt file from within Wordpress.

thanks. Do you know how I can check if the htaccess code worked?

 

[jmcgoose]

thanks. Do you know how I can check if the htaccess code worked?

build some backlinks and see if they show up

 

[JustUs]

build some backlinks and see if they show up

Change your user agent to a string incorporating the string of a banned bot and try to access the site.

Example:
banned: *mj12bot*
change UA to micemj12bot2 and access the site.

 

[thelartman]

You can check whether you site is blocking bots on wanna browser


By using the majestic user agent (look it up cos cant post the URL in here) it replicates the majestic bot.

I have found in recent weeks that blocking bots with .htaccess has stopped working and I have you use a plugin, which id rather not.

Does anyone have any definite working code?