Fastest way to harvest relevant URLs

postcd

BANNED

Nov 14, 2009

145

8

• Feb 23, 2010

• #1

Hi,

Im using URL Harvester 2008 to harvest URLs from the net.

It saves URLs from Google, but the problem is that google ban you when accessing it too many times per time period.

So my question is what software or method You are using to harvest URLs from the net. What is the fastest method?

Thank you much

 

J

jesdon2

Junior Member

Jun 19, 2009

122

85

• Feb 23, 2010

• #2

Buy Scrapebox. You can scrape thousands of urls, with custom footprints, using random proxies and then remove all duplicates and sort the list by pagerank. Very useful tool.

 

L

liangtvdotcom

Newbie

Dec 27, 2009

49

0

• Feb 23, 2010

• #3

For example I wana scrape all the URLs links from www. xxx . com what keyword should i type?

"www. xxx . com /urls/"
or
site:www . xxx .com / urls ?

using scapebox? i tried alot of ways but cant get thousands of them

 

keverone

Junior Member

Oct 4, 2009

151

47

• Feb 23, 2010

• #4

jesdon2 said:

Buy Scrapebox. You can scrape thousands of urls, with custom footprints, using random proxies and then remove all duplicates and sort the list by pagerank. Very useful tool.

Click to expand...

You can ... just go get scrapbox it is sold on BHW search it out..

 

ericsson

Elite Member

Apr 25, 2009

2,640

8,233

• Feb 23, 2010

• #5