Refresh your robots.txt knowledge

Although robots.txt files seem like an easy to understand topic, in reality it's one of the more complex topics within Technical SEO. Why? For starters, because robots.txt files influence crawl budget, duplicate content and JavaScript rendering. What's more, different search engines interpret robots.txt differently.

The guys at ContentKing have put together the ultimate guide to help you get your robots.txt right.

Source: https://www.contentkingapp.com/academy/robotstxt/

Create robots.txt file online and upload it on root folder.

Yuga said:

Create robots.txt file online and upload it on root folder.

Click to expand...

I tried that and it broke my windows 10!