1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to scrape top 10 results in Google in bulk?

Discussion in 'Black Hat SEO' started by ardley216, Aug 24, 2014.

  1. ardley216

    ardley216 Elite Member

    Joined:
    Mar 28, 2008
    Messages:
    2,391
    Likes Received:
    2,356
    Occupation:
    Finding easy keywords
    Location:
    1,500,000,000 Keywords Re
    Home Page:
    Hi,

    Basically as the title says, is there any way to scrape the top 10 urls for search terms on mass?

    Or if anyone knows a service that supplies this?

    Thanks!
     
  2. ticalico

    ticalico Registered Member

    Joined:
    Oct 10, 2011
    Messages:
    53
    Likes Received:
    15
    Occupation:
    YouTube views provider, SEO and Google Analytics A
    Hmm...

    You could maybe right a program in php and use CURL.

    It would scrape the whole page and you could have it return JUST content of a specific ID or Class.

    Target those top 10 ignoring or not ignoring the advertisements also listed in the page.

    Put it in a list etc....
     
  3. Schvamp

    Schvamp Power Member

    Joined:
    Feb 13, 2012
    Messages:
    684
    Likes Received:
    549
    Location:
    Hogwarts
    I'm sure it can be done with scrapebox, with the right settings, unfortunately I ain't that familiar with it tho.
    If you need a custom scraper, shoot me a PM.
     
  4. himanuzo

    himanuzo Supreme Member

    Joined:
    Feb 10, 2010
    Messages:
    1,342
    Likes Received:
    277
    Location:
    Asia
    Home Page:
    Market Samurai software can do it.
     
  5. ardley216

    ardley216 Elite Member

    Joined:
    Mar 28, 2008
    Messages:
    2,391
    Likes Received:
    2,356
    Occupation:
    Finding easy keywords
    Location:
    1,500,000,000 Keywords Re
    Home Page:
    Hi,

    Thanks for your suggestions.

    I will be needing this to scrape tens of thousands of search results everyday.
     
  6. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,559
    Likes Received:
    4,315
    Location:
    Toronto
    Home Page:
    • Thanks Thanks x 2
  7. himanuzo

    himanuzo Supreme Member

    Joined:
    Feb 10, 2010
    Messages:
    1,342
    Likes Received:
    277
    Location:
    Asia
    Home Page:
    For this, you need to use many paid proxies monthly. Because Google doesn't allow it.
     
  8. macdonjo3

    macdonjo3 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 8, 2009
    Messages:
    5,559
    Likes Received:
    4,315
    Location:
    Toronto
    Home Page:
    Actually, with proper intervals, he won't need many at all. I tested this last year... unless things have changed since then.
     
    • Thanks Thanks x 1
  9. Pejlis

    Pejlis Regular Member

    Joined:
    Jul 7, 2013
    Messages:
    210
    Likes Received:
    20
    you cant get thousands they dont allow you
     
  10. Schvamp

    Schvamp Power Member

    Joined:
    Feb 13, 2012
    Messages:
    684
    Likes Received:
    549
    Location:
    Hogwarts
    Lets say OP needs to scrape 30,000 urls a day.
    30000 / 10 results per page = 3000 requests.
    3000 / 24h a day = 125 requests per hour.
    125 / 60 minutes per hour = ~2 requests/min.

    2 requests per minute, not possible?
    Easy to scale with proxy rotation,vps and multithreading.
     
    • Thanks Thanks x 3
    Last edited: Aug 24, 2014
  11. ardley216

    ardley216 Elite Member

    Joined:
    Mar 28, 2008
    Messages:
    2,391
    Likes Received:
    2,356
    Occupation:
    Finding easy keywords
    Location:
    1,500,000,000 Keywords Re
    Home Page:
    Thanks for that. How does it work? Luckily my business partner is learning python right now.

    Yeah that sounds good. Except, I'll be doing 300k a day.

    Is there a php script that uses CURL, like ticalico mentioned? One that does proxy rotation?
     
  12. SeoQuest

    SeoQuest Junior Member

    Joined:
    Jun 24, 2012
    Messages:
    142
    Likes Received:
    41
    Location:
    Europe
    In Scrapebox in Results field type 10 and it will scrape the first 10 urls for each keyword in your list.
     
    • Thanks Thanks x 1
  13. prab1996

    prab1996 Elite Member

    Joined:
    Jan 8, 2013
    Messages:
    3,496
    Likes Received:
    2,027
    Occupation:
    your gf's <3 ♥♥♥♥
    Location:
    Prab1996.com
    Home Page:
    this can be easily done with sb.

    if you can't pay then try gscraper free and set it to scrape only top 10 results.
    -=-
     
  14. mindmaster

    mindmaster Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 16, 2010
    Messages:
    2,501
    Likes Received:
    1,135
    Location:
    at my new office
    Yes, Scrapebox can do that easily just as SeoQuest and Prab said.
     
  15. MissioneR

    MissioneR Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 8, 2012
    Messages:
    1,562
    Likes Received:
    156
    Location:
    In my GUTS
    It's the simple method where you can get what you want.
     
  16. ardley216

    ardley216 Elite Member

    Joined:
    Mar 28, 2008
    Messages:
    2,391
    Likes Received:
    2,356
    Occupation:
    Finding easy keywords
    Location:
    1,500,000,000 Keywords Re
    Home Page:
    Yes, except I would need it to be put straight into a database. and I am looking at 6000 requests day.