mirrorer
Elite Member
- Jan 30, 2009
- 2,124
- 2,530
https://www.zerohedge.com/technology/openai-launches-web-crawler-gptbot-amid-plans-gpt-5
Modify your robots.txt and add:
More about how to block it here: https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
Cheers
Modify your robots.txt and add:
Code:
User-agent: CCBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: GPTBot
Disallow: /More about how to block it here: https://neil-clarke.com/block-the-bots-that-feed-ai-models-by-scraping-your-website/
Cheers
