• On Wednesday, 19th February between 10:00 and 11:00 UTC, the forum will go down for maintenance. Read More

Google Using RSS/Atom Feeds To Discover New URLs

sokol

Registered Member
Joined
Sep 30, 2008
Messages
56
Reaction score
9
Google announced a new feature last week that uses RSS and Atom feeds for the discovery of new webpages.

Using feeds for discovery allows Google to index new pages more quickly than traditional crawling methods. According to the announcement, Google may use ?many potential sources to access updates from feeds including Reader, notifications services, or direct crawls of feeds.

If you?re publishing an RSS/Atom feed, make sure that those files are not disallowed by your robots.txt file. If you?re unsure whether your feeds are crawlable, use the robots.txt tester in Google Webmaster Tools.
http://www.websitemagazine.com/cont...sing-rss-atom-feeds-to-discover-new-urls.aspx
 
Back
Top