Scrape Box Harvesting..

hi,

i am new to it.. i saw we have a limit of only 1000 results, then how can i harvest 1k+ urls? is it done by adding more keywords? if so, if i select custom footprint and insert keyword in keyword box, will it work?

you can combine a footprint with a keyword, at the end you will have for each keyword at least 1k URLs that will contain also the footprint. so if you add 100 keyword and combine them with a footprint you will have 100K URLs. This is the ideal case, in reality you will have a little bit lower amount but it will be close...

Correct. Just set the results to 1000 and they will be the max per keyword.

If you use a huge keyword list (450k) it should look like this:



Just get some good google safe proxies, at least 50. or non google safe if you only use bing/yahoo.

After scraping i usually end up with 4+ million URLs. Many of them duplicate if using more than one search engine but still enough.