Tool to extractURL's from G Search Results

[supahoopsa]

Have been searching around for a while to find a free (or cracked) piece of software that will extract URL's from a G search.

So if I search for "great red widgets" it will do a search in Google for that KW phrase and then extract the URL's from the first x number of listings to a text or csv file.

I have seen various tools that do the first 100 but need a lot more than that.

Thanks in advance
David

 

[undeterminederror]

First: you set google show 100 results/page

Then you have to install 2 mozilla plugins:
-autopager mozila plugin
-snap links plus mozila plugin

with the first plugin you will see 1000 google results.
with the second plugin you will select the results with the mouse right button then clic ctrl -> copy ; open a txt file then paste.

ta - daaa: 1000 links in less than 1 min

 

[supahoopsa]

Thanks, great suggestion

Any others ?

 

[blackhaty]

try scrape genius.

 

[supahoopsa]

try scrape genius.

Thanks, that s/w looks exactly what I was after.

Downloaded okay but email registration appears to be broken

 

[mailcold]

First: you set google show 100 results/page

Then you have to install 2 mozilla plugins:
-autopager mozila plugin
-snap links plus mozila plugin

The URL for snap links plus mozila plugin is

snaplinks.mozdev.org

 

[orangeblossoms]

First: you set google show 100 results/page

Then you have to install 2 mozilla plugins:
-autopager mozila plugin
-snap links plus mozila plugin

with the first plugin you will see 1000 google results.
with the second plugin you will select the results with the mouse right button then clic ctrl -> copy ; open a txt file then paste.

ta - daaa: 1000 links in less than 1 min

Damn this is awesome and so simple. With snap links though, when I paste it into notepad, there is no line break between each url. How do I get the urls on each separate line?

EDIT:
Okay I figured it out that it's "\n" for line break, but not all urls have the same ending. Any easy way to move all links onto their own line?

 

[JohnKarr]

just use lazerURL

 

[Debian]

Link Whore for sure. Best by far!

 

[markdigerati]

You can use http://www.digeratimarketing.co.uk/2008/12/16/curl-page-scraping-script/

and bung <cite> in it. Since you have the code you can just whack it right into a csv or whatever you want.

 

[44blackhatters]

The firefox plugins suggested are great...these will be very helpful. Thanks.

 

[undeterminederror]

Okay I figured it out that it's "\n" for line break, but not all urls have the same ending. Any easy way to move all links onto their own line?

instead adding line break after the link add it in front the link, in front the http





other twists:

with the search->replace function you can clean scraped lists like seobook most searched terms or google images links, actually, scrapping google images is pretty damn good for seo and traffic(for the ones who know to use them).
you can use search-replace from office word, wordpad or notepad++

 

[orangeblossoms]

instead adding line break after the link add it in front the link, in front the http

Lol I feel stupid now. This calls for an appropriate emoticon. :smashfrea
Thanks for the help and twists! I like your scraping method, it's nice and fast.