Google search url's SCRAPING HOW TO??

5

5thday

Newbie
Joined
Jun 20, 2008
Messages
11
Reaction score
0
hi

so i need to scrape google search results but i only need links/urls from these results.I am using some search operators like inurl: and i get like 1500 results.Now, what i wanna do is to get urls from these 1500results,no need for anything else, just urls.There must be some simple script or app that can do that...Any ideas?

thanks for any suggestion:cool:
 
Advertise on BHW
use the expired domain software. It scapes urls, just copy and paste
 
5thday said:
hi

so i need to scrape google search results but i only need links/urls from these results.I am using some search operators like inurl: and i get like 1500 results.Now, what i wanna do is to get urls from these 1500results,no need for anything else, just urls.There must be some simple script or app that can do that...Any ideas?

thanks for any suggestion:cool:
Click to expand...

You could use PHP with CURL.

Check here:
Code: 
http://www.devnewz.com/devnewz-3-20041221UsingPHPCURLLibrarytoScrapetheInternet.html
 
hi guys

thanks for ultrafast response and suggestions .i am pretty noobish when it comes to php and curl so cannot handle this one myself yet:)

snowwhite, this expired domain software sound like smth i am looking for now but google gives me few "expired domains software" so i don't know which one you have in mind?Please could you clarify?
this one:
hxxp://www.expireddomainsleuth.com/
or this one
hxxp://www.domainsoftware.org/
or this one?
http://hxxp://www.dnware.com/

thanks again for your help
 
Last edited:
if you're coding it in PHP, just use regex to grab the URL's in the links, its easy. If you need help PM me and ill make it for you if i have the time.
 
hi again

thanks a lot snowwhite for the link, i ve just tested it and unfortunately this tool doesn't do exactly what i need.It can scrape google but the output are just domain names like outputdomain.com and not full URL's like outputdomain.com/happyscarping.php which i need.anyway thanks for your time;)

If anyone have any suggestion i am still looking for some ideas.I can't code myself so i am looking for smth already made.There are some tools i have found but they scrape just max 100 results from google and i need like few thousands.For better understanding just have a look at the link below.Basically i wanna get smth very very similar but with all the results available
:)

hxxp://goohackle.com/scripts/google_parser.php

thank you
 
5thday said:
hi

so i need to scrape google search results but i only need links/urls from these results.I am using some search operators like inurl: and i get like 1500 results.Now, what i wanna do is to get urls from these 1500results,no need for anything else, just urls.There must be some simple script or app that can do that...Any ideas?

thanks for any suggestion:cool:
Click to expand...

Try hxxp://3w.planetsourcecode.c0m

They have a lot of scripts under the PHP section that can do this sort of thing and they're free

I recall one script that I think was called 'spider' and it would scrape links or images or whatever you need. Give it a try.
 
I am using Web Scraper Plus+ for this kind of scraping, no code skills needed.
 
search for serp harvester in download section, there is a free tool
 
Web Scraper Plus+

HTML: 
h**p://filestube.com/3db0f942551befea03ea/details.html


Pass: AdaMskIvelWalenty
 
Last edited:
timestimes said:
Web Scraper Plus+

HTML: 
h**p://filestube.com/3db0f942551befea03ea/details.html
Pass: AdaMskIvelWalenty
Click to expand...

Needs a password to unzip file....

edit: my bad, the password is right in the post.
 
Last edited:
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top