Any updated/working code to block ahref/majestic bots?
- Thread starter beingink
- Start date
DigitalCon
Power Member
- Jul 27, 2014
- 538
- 99
This works good for all my PBN domains, hope it works for you too.
Code:
SetEnvIfNoCase User-Agent .*rogerbot.* bad_bot
SetEnvIfNoCase User-Agent .*exabot.* bad_bot
SetEnvIfNoCase User-Agent .*mj12bot.* bad_bot
SetEnvIfNoCase User-Agent .*dotbot.* bad_bot
SetEnvIfNoCase User-Agent .*gigabot.* bad_bot
SetEnvIfNoCase User-Agent .*ahrefsbot.* bad_bot
SetEnvIfNoCase User-Agent .*sitebot.* bad_bot
SetEnvIfNoCase User-Agent .*semrushbot.* bad_bot
SetEnvIfNoCase User-Agent .*ia_archiver.* bad_bot
SetEnvIfNoCase User-Agent .*searchmetricsbot.* bad_bot
SetEnvIfNoCase User-Agent .*seokicks-robot.* bad_bot
SetEnvIfNoCase User-Agent .*sistrix.* bad_bot
SetEnvIfNoCase User-Agent .*lipperhey spider.* bad_bot
SetEnvIfNoCase User-Agent .*ncbot.* bad_bot
SetEnvIfNoCase User-Agent .*backlinkcrawler.* bad_bot
SetEnvIfNoCase User-Agent .*archive.org_bot.* bad_bot
SetEnvIfNoCase User-Agent .*meanpathbot.* bad_bot
SetEnvIfNoCase User-Agent .*pagesinventory.* bad_bot
SetEnvIfNoCase User-Agent .*aboundexbot.* bad_bot
SetEnvIfNoCase User-Agent .*spbot.* bad_bot
SetEnvIfNoCase User-Agent .*linkdexbot.* bad_bot
SetEnvIfNoCase User-Agent .*nutch.* bad_bot
SetEnvIfNoCase User-Agent .*blexbot.* bad_bot
SetEnvIfNoCase User-Agent .*ezooms.* bad_bot
SetEnvIfNoCase User-Agent .*scoutjet.* bad_bot
SetEnvIfNoCase User-Agent .*majestic-12.* bad_bot
SetEnvIfNoCase User-Agent .*majestic-seo.* bad_bot
SetEnvIfNoCase User-Agent .*dsearch.* bad_bot
SetEnvIfNoCase User-Agent .*blekkobo.* bad_bot
SetEnvIfNoCase User-Agent .*screaming frog seo spider/*.* bad_bot
SetEnvIfNoCase User-Agent .*PHPCrawl.* bad_bot
SetEnvIfNoCase User-Agent .*gocrawl.* bad_bot
SetEnvIfNoCase User-Agent .*DigExt.* bad_bot
SetEnvIfNoCase User-Agent .*DomainSONOCrawler.* bad_bot
SetEnvIfNoCase User-Agent .*TweetmemeBot.* bad_bot
SetEnvIfNoCase User-Agent .*OpenHoseBot/2.1.* bad_bot
SetEnvIfNoCase User-Agent .*Kraken/0.1.* bad_bot
SetEnvIfNoCase User-Agent .*-Java-.* bad_bot
SetEnvIfNoCase User-Agent .*ubermetrics.* bad_bot
SetEnvIfNoCase User-Agent .*best-seo.* bad_bot
SetEnvIfNoCase User-Agent .*Synapse.* bad_bot
SetEnvIfNoCase User-Agent .*Harvest.* bad_bot
SetEnvIfNoCase User-Agent .*Harvester.* bad_bot
SetEnvIfNoCase User-Agent .*harvester.* bad_bot
SetEnvIfNoCase User-Agent .*harvest.* bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
pauverinox
Newbie
- Apr 21, 2016
- 18
- 1
This is for htaccess right?
Where do I insert that code on my wordpress htaccess.
Where do I insert that code on my wordpress htaccess.
Code:
# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPressDigitalCon
Power Member
- Jul 27, 2014
- 538
- 99
Yes, this for .htaccess.
Paste if after the default wordpress .htaccess code.
After this line
# END WordPress
Ashk881
Elite Member
- Aug 24, 2021
- 1,695
- 2,072
Nginx doesn't have .htacaess support. You will need to convert your htacess file to a nginx config block.
Heres a converter https://winginx.com/en/htaccess
Not sure how well it works tho.