If anyone can suggest something to make this it will be appreciated.
I know I can use a robots.txt file but not all bots respect this, I am sure 100% of this.
There's documentation by Google to prevent Google bots to access your website, but I am also 100% sure that their documentation is not...
Deleted member 1786666
Thread
block bot traffic
block bots
blockseocrawlers
crawl
google bot
a small plugin (2.40kb) that writes code to .htaccess to block seo crawlers.
user agents that plugin blocks:
rogerbot - opensiteexplorer
Dotnetdotcom.org - Moz
mj12bot - Majestic
ahrefsbot - ahrefs
exabot
gigabot
sitebot
dot bot
virus total...
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.