pizzafrombonita19
Registered Member
- Jun 17, 2019
- 61
- 6
The error I get from Ahrefs: Ahrefs’ Site Audit bot couldn’t access the URL you want to start crawling from. You won’t be able to complete a site audit, but you can continue to set this project up for other tools. The HTTP server returned error 403: "Forbidden". This request was likely filtered by server configuration.
The error from Moz: Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. Update these tags to allow your page and the rest of your site to be crawled. If this error is found on any page on your site, it prevents our crawler (and some search engines) from crawling the rest of your site.
I don't have robots.txt file so im wondering what coud be the reason for this.
The error from Moz: Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. Update these tags to allow your page and the rest of your site to be crawled. If this error is found on any page on your site, it prevents our crawler (and some search engines) from crawling the rest of your site.
I don't have robots.txt file so im wondering what coud be the reason for this.