Recent content by Tom2018

  1. T

    Majestic robot blocked but still crawling my website

    Hi, is there an update on this issue? As far as I see blocking IP ranges is the best solution? What do you guys think? Just a thought: Usually when we block we trigger a 403 error. What if this 403 triggers a revisit with an "stealth" user agent. So if we just show a blank page (like...
Back
Top