1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

scraping/harvesting urls from the serps

Discussion in 'Black Hat SEO' started by bizbez, Feb 28, 2011.

  1. bizbez

    bizbez Registered Member

    Joined:
    Jan 26, 2011
    Messages:
    86
    Likes Received:
    3
    hi guys,

    wanted to ask you how do you scrape urls from google and other SE.
    i use a firefox plugin with a script that can scrape up to 100 urls per query. but it has some limits for scraping other languages than english.

    i saw this tool, and would like to know if anyone can recommend.
    Code:
    serpscraper.com/
    
    also, can scrapebox scrape urls from other search engines like yandex, and baidu? can it scrape urls from google cc-TLD (google.de)?

    Thanks
     
  2. chris456

    chris456 Regular Member

    Joined:
    May 17, 2010
    Messages:
    281
    Likes Received:
    567
    Scrapebox can scrape Urls from all Google Tlds (.ch , .de , .at etc ), but Baidu , Yandex and others as far as I know it coudn't .

    SerpScraper is a little bit expensive for me to use it for only 1 purpose , but it should work nice and maybe even better than scrapebox, I have Scrapebox so I don't need it , but it is on DNL list here anyway , so we can only dream about it -:) , there is also another product scrapegenius , but I have never tried too , it costs less money , try to google for that.
     
  3. cyberzilla

    cyberzilla Elite Member Premium Member

    Joined:
    Nov 15, 2009
    Messages:
    2,204
    Likes Received:
    3,363
    Location:
    zeta reticuli
    Scrapebox is the BEST tool for URL scraping. SB is very cheap when compared to serpscraper! Also it has got more features & add-ons which is lacking in serpscraper. SB scrape URLs only from G0ogle, Yah0o, B1ng & A0l. Most of the SB users use just Yahoo & G0olge for scraping as the search results and indexing is more here. Also you don't really have to worry about yandex and baidu as they are not a major SE.
     
  4. cookie48

    cookie48 Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 19, 2009
    Messages:
    128
    Likes Received:
    14
    Why you dont create your own tool for scraping?
     
  5. cyberzilla

    cyberzilla Elite Member Premium Member

    Joined:
    Nov 15, 2009
    Messages:
    2,204
    Likes Received:
    3,363
    Location:
    zeta reticuli
    Well, not everyone's gifted with Programming knowledge right.
     
  6. chris456

    chris456 Regular Member

    Joined:
    May 17, 2010
    Messages:
    281
    Likes Received:
    567
    Does your question means that you are offering your programming abilities to BHW community ? I agree with Cyberzilla that not everybody is able to create such a simple tool like a Google scraper.