URL harvester requied

iglow

Elite Member

Feb 20, 2009

2,254

870

• Mar 18, 2009

• #1

Any good working url harvesters? Seems none of previous works..
Mainly i wanna put some goolge search site query in, and harvest urls.

 

kouxx

Junior Member

Aug 2, 2008

160

39

• Mar 18, 2009

• #2

hey, this is your 100th post :rock:
there is the bhw url harvester in the jr vip section...

 

Last edited: Mar 18, 2009

S

skweekykleen

Newbie

Jul 8, 2008

47

68

• Mar 18, 2009

• #3

not sure if it fits your needs for the exact task you've got at hand, but one little tool that I like is called "Link Gopher" -- it's a firefox plugin.

PHP:

https://addons.mozilla.org/en-US/firefox/search?q=link+gopher&cat=all

It puts a little button at the bottom right-hand side of your browser window, when you click it, it scrapes all of the links from the page you are on, then opens a new tab with them all organized, de-duped, etc.

When I am just trying to grab a few hundred links for something, I usually just set Google to return 100 results per page, then use this tool to scrape 3-4 pages worth and I'm off...

Hope it helps...

 

iglow

Elite Member

Feb 20, 2009

2,254

870

• Mar 19, 2009
• Thread Starter Thread Starter

• #4

kouxx said:

hey, this is your 100th post :rock:
there is the bhw url harvester in the jr vip section...

Click to expand...

i do not see junior section even thought i have junior next to my nick. where should i find it or who to msg?

 

F

forgot

Senior Member

May 3, 2008

927

713

• Mar 19, 2009

• #5

kouxx said:

hey, this is your 100th post :rock:
there is the bhw url harvester in the jr vip section...

Click to expand...

How did you come to know about it?

You are not Jr VIP...

iglow said:

i do not see junior section even thought i have junior next to my nick. where should i find it or who to msg?

Click to expand...

I guess it is in Jr VIP Section.

You need to be Jr VIP to see the section.

Quote:

Jr. VIP Membership

If you have been a member of the forum for at least 60 days, and have made at least 100 posts, your $97 donation will promote you to a Jr. VIP membership, which allows you to access the hidden Jr. VIP forums and downloads sections.

 

iglow

Elite Member

Feb 20, 2009

2,254

870

• Mar 19, 2009
• Thread Starter Thread Starter

• #6

Ah that is how it works. So the only way to get to top is $$$

 

R

ravi72194

Newbie

Apr 6, 2007

11

1

• Mar 19, 2009

• #7

i have made a script to harvest url just search the particular word and it harvest url from google. but this script is not free . if interested then let me know .
i will show you DEMO.

 

dekadent30

Elite Member

Nov 10, 2008

1,910

1,036

• Mar 26, 2009

• #8

try SerpScraper or global squid se parser

 

rap8557

Banned - Selling via PM

Sep 15, 2008

2,000

631

• Mar 27, 2009

• #9

try this UMAX this is the one that i've used in harvesting but it is russian language

http://www.blackhatworld.com/blackhat-seo/black-hat-seo/33813-harvesting-urls.html

Click to expand...

 

S

solobeton

Junior Member

Feb 24, 2008

105

86

• Apr 19, 2009

• #10

rap8557 said:

try this UMAX this is the one that i've used in harvesting but it is russian language

Click to expand...

According to me this is the best scraper i ever use.
I'm using the demo version as well but i can still get 1000's and 1000's of urls.

It is in russian, i don't know how to buy the full version.