Bot crawlers blocked on PBN
- Thread starter patlang
- Start date
DigitalCon
Power Member
- Jul 27, 2014
- 538
- 99
Its been shared on BHW quite a few times. You could search for it rather than creating a thread.
Anyways, here:
Anyways, here:
Code:
SetEnvIfNoCase User-Agent .rogerbot. bad_bot
SetEnvIfNoCase User-Agent .exabot. bad_bot
SetEnvIfNoCase User-Agent .mj12bot. bad_bot
SetEnvIfNoCase User-Agent .dotbot. bad_bot
SetEnvIfNoCase User-Agent .gigabot. bad_bot
SetEnvIfNoCase User-Agent .ahrefsbot. bad_bot
SetEnvIfNoCase User-Agent .sitebot. bad_bot
SetEnvIfNoCase User-Agent .semrushbot. bad_bot
SetEnvIfNoCase User-Agent .ia_archiver. bad_bot
SetEnvIfNoCase User-Agent .searchmetricsbot. bad_bot
SetEnvIfNoCase User-Agent .seokicks-robot. bad_bot
SetEnvIfNoCase User-Agent .sistrix. bad_bot
SetEnvIfNoCase User-Agent .lipperhey spider. bad_bot
SetEnvIfNoCase User-Agent .ncbot. bad_bot
SetEnvIfNoCase User-Agent .backlinkcrawler. bad_bot
SetEnvIfNoCase User-Agent .archive.org_bot. bad_bot
SetEnvIfNoCase User-Agent .meanpathbot. bad_bot
SetEnvIfNoCase User-Agent .pagesinventory. bad_bot
SetEnvIfNoCase User-Agent .aboundexbot. bad_bot
SetEnvIfNoCase User-Agent .spbot. bad_bot
SetEnvIfNoCase User-Agent .linkdexbot. bad_bot
SetEnvIfNoCase User-Agent .nutch. bad_bot
SetEnvIfNoCase User-Agent .blexbot. bad_bot
SetEnvIfNoCase User-Agent .ezooms. bad_bot
SetEnvIfNoCase User-Agent .scoutjet. bad_bot
SetEnvIfNoCase User-Agent .majestic-12. bad_bot
SetEnvIfNoCase User-Agent .majestic-seo. bad_bot
SetEnvIfNoCase User-Agent .dsearch. bad_bot
SetEnvIfNoCase User-Agent .blekkobo. bad_bot
SetEnvIfNoCase User-Agent .screaming frog seo spider/.* bad_bot
SetEnvIfNoCase User-Agent .PHPCrawl. bad_bot
SetEnvIfNoCase User-Agent .gocrawl. bad_bot
SetEnvIfNoCase User-Agent .DigExt. bad_bot
SetEnvIfNoCase User-Agent .DomainSONOCrawler. bad_bot
SetEnvIfNoCase User-Agent .TweetmemeBot. bad_bot
SetEnvIfNoCase User-Agent .OpenHoseBot/2.1. bad_bot
SetEnvIfNoCase User-Agent .Kraken/0.1. bad_bot
SetEnvIfNoCase User-Agent .-Java-. bad_bot
SetEnvIfNoCase User-Agent .ubermetrics. bad_bot
SetEnvIfNoCase User-Agent .best-seo. bad_bot
SetEnvIfNoCase User-Agent .Synapse. bad_bot
SetEnvIfNoCase User-Agent .Harvest. bad_bot
SetEnvIfNoCase User-Agent .Harvester. bad_bot
SetEnvIfNoCase User-Agent .harvester. bad_bot
SetEnvIfNoCase User-Agent .harvest. bad_bot
<Limit GET POST HEAD>
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Limit>
shieldsroadfirm
Newbie
- Oct 12, 2016
- 8
- 1
Asked, shared and reshared multiple times already. There are many threads with that information as well as on the best ways of using them.
patlang
Regular Member
- Sep 3, 2015
- 220
- 38
Imagine I am on my mobile so would take me years to find a proper answer. Much more convenient was to create a new thread which apparently was not useful just for me
- Dec 13, 2010
- 4,875
- 3,317
Considering you'll need a computer to put this in the htaccess, I suppose you have a computer, then you can search.
patlang
Regular Member
- Sep 3, 2015
- 220
- 38
Thank you for pumping the post up with useless posts however I already got the answer.
Farewell
- Dec 13, 2010
- 4,875
- 3,317
Yes please lecture me about posting for nothing... How ironic.