TestingViews
Registered Member
- May 11, 2019
- 74
- 29
is this ok to disallow /feed/ in robots.txt, would that change anything in google perspective, because there are so many /feed/ going on in google bots lately
Disallow /feed/ robots.txt
- Thread starter TestingViews
- Start date
Roger Marquez
Elite Member
- Apr 17, 2017
- 5,126
- 7,816
Indeed!
Those pages have close to zero SEO value and Googlebot won´t often index them so there´s no need to waste crawl budget on those pages. This is why you normally see them inside the Exclude > Crawled — Currently Not Indexed tab.
I often disallow them like this...
Disallow /feed/
Disallow /*/feed/
Disallow /*/*/feed/
Disallow /*/*/*/feed/
Disallow /*/*/*/*/feed/
Disallow /*/*/*/*/*/feed/
On a different note, if you have a small website, this won´t be such an issue but, if you´re OCD like me (lol), I often try to set a solid foundation for the future.
Take the time to read this article as it has some nice insights on the topic...
https://www.webnots.com/why-and-how-to-disable-wordpress-feeds/
Cheers!!
Those pages have close to zero SEO value and Googlebot won´t often index them so there´s no need to waste crawl budget on those pages. This is why you normally see them inside the Exclude > Crawled — Currently Not Indexed tab.
I often disallow them like this...
Disallow /feed/
Disallow /*/feed/
Disallow /*/*/feed/
Disallow /*/*/*/feed/
Disallow /*/*/*/*/feed/
Disallow /*/*/*/*/*/feed/
On a different note, if you have a small website, this won´t be such an issue but, if you´re OCD like me (lol), I often try to set a solid foundation for the future.
Take the time to read this article as it has some nice insights on the topic...
https://www.webnots.com/why-and-how-to-disable-wordpress-feeds/
Cheers!!