InsanelySane
Power Member
- Nov 23, 2013
- 574
- 116
How many people are doing this for their sites? Im only interested in blocking via robots.txt. However, I've heard ahrefs and majestic tend to not honor your decision.
Once Ahrefs bot is blocked in robots.txt, we stop crawling the site and drop blocked pages from the index altogether, but we will still continue collecting and showing the links pointing to this site from other domains.
user-agent: AhrefsBot
disallow: /