I have a blog with a very huge amount of comment (>50,000 pages/entries) that has built up over the last year or so. Right now only about 6500 pages are indexed in Google. I would like more pages to be indexed. While I understand to get all of them indexed is not realistic, I would like at least 50% to be indexed. Is there a plugin that will crawl your blog and then slowly ping each entry or page/entry over time? I searched and have not seen anything like that. All of the pinger stuff is for when you write it to not excessively ping. I am looking to just start to grab maybe 100 of them per day and have something ping the various services in hopes more pages get indexed. Either that or I am up for any other methods anyone might have to try to increase site indexes. I have the RSS feed submitted all over as well, but perhaps this can be used in a better way. thanks for anyone with any ideas here.
The google XML Sitemap plugin would be a good start.. at least google will know where all your pages are.
yep, I have been using that from day one. The problem now is the site is so big if I tell it to index all pages it crashes/has issues, so I have limited it to the last 1000 posts/pages. I think what is needed is an external sitemap tool that could map all pages on the site, then I can upload that to the site for a bit and turn off the WP one. Anyone know of a decent tool that can do this?
