Scraping google results, getting PR and HTML-code. I need your expert opinion on this!

tinkerbellO

Newbie
Joined
Aug 30, 2012
Messages
3
Reaction score
1
Hello friends! I need an expert opinion on the following problem.


In an university project I have find a solution to do the following tasks:

- I need to scrape google search results. Mainly I want 100.000 keywords and the first 50 search results (5 pages) exported (only the URLs and the ranking of the domain with the current keyword).

- Then I have to check the PR of all 5.000.000 websites (duplicates included and only the first page).

- After that I need the HTML-code of all 5.000.000 websites (duplicates included and only the first page).

- After the job is finished, I want the results to be exported (txt, csv, ...) in order to examine the results elsewhere

- This job should be done automatically and periodically without user interaction (24/7 would be the best case)


QUESTIONS

* Is this possible with scrapebox or can you recommend other tools? I also can write my own java program for this purpose.

* Should I use private or public proxies to finish this task? How many of the proxies do I need?



Thanks in advance for your expert opinion.
tinkerbellO
 
Back
Top