help! I scraped 500,000 posts using wordpress/feedwordpress and exported/converted/uploaded them a category at a time into a new blogger blog spot blog. google will only visit 2000 times a day or 20Megs in total bandwidth per day it seems. while indexed pages are going up by a couple thousand a day using site:site.blogspot.com only 30 pages a day are showing up under top pages. is there any way to speed this process up?

no one has any idea how to speed up page recognition process in wmt?

Have you thought about building backlinks to the pages that are not indexed?

is there really a limit of 2000 visits/day? is it only to blogspot?

Originally Posted by Rushdie

is there really a limit of 2000 visits/day? is it only to blogspot?

He's talking about Google's robot for indexing, he say's it comes around about 2k times per day.

Originally Posted by spsfinest

He's talking about Google's robot for indexing, he say's it comes around about 2k times per day.

yes this seems correct a few weeks later. im not sure yet if its a blogger blogspot only thing so i copied and pasted it to a few dozen new free web 2.0's and see soon hopefully! hope i can get around this technical hurdle so i can start releasing larger websites on free web 2.0s! =)

Try posting more unique content.

Use profile backlinks or whatever you can get in masses and deeplink all the content on your blog, it will help (a lot) getting everything indexed.

Originally Posted by n3wbseo

help! I scraped 500,000 posts using wordpress/feedwordpress and exported/converted/uploaded them a category at a time into a new blogger blog spot blog. google will only visit 2000 times a day or 20Megs in total bandwidth per day it seems. while indexed pages are going up by a couple thousand a day using site:site.blogspot.com only 30 pages a day are showing up under top pages. is there any way to speed this process up?

You can consider to increase bot visit rate in google Webmaster Tool

Yes you can increase it...but it doesn't improve more than 10-15% over the standard.

I have a similar project that targets millions of keywords in a fully automated system creating keywords and dynamic PHP content pages from a huge database of spintax articles that dynamically insert my target keywords.

In two months, the google bot has hit the domain over 300,000 times...but only about 50K pages are indexed when I search site:domain.com

Might be a quality issue...as the quality is most certainly low. I only set it up for testing purposes. If it generates revenue someday that is great. But I just wanted to see what it would do.

Another thought I had (but I haven't tested) is splitting a huge site into 10K page chunks via subdomain or cheap .info sites. site1.domain.com site2.domain.com and doing some blasts to each. Technically these should be seen as separate sites and could all be indexed at 2K/day rate max. Then once your page are all indexed, 301 redirect each of your subdomains to your main site and insert the articles.

Good luck on your project

Originally Posted by thachcaodotcom

You can consider to increase bot visit rate in google Webmaster Tool

i added 2000 links, now my blog disappear on Google, how can i solve this problem my blog: xuonginoffset.blogspot.com, pls give me some advise

just a quick follow up: google starts indexing more pages faster over time without any backlinks to atleast a sub domain like blogspot.com.

You can start building backlinks.

You can change it here:

Code:

http://www.google.com/webmasters/