1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[TIP] Scrapebox + Google Banned Proxies + Google API

Discussion in 'Black Hat SEO Tools' started by Buzzika, May 23, 2016.

  1. Buzzika

    Buzzika Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2009
    Messages:
    1,161
    Likes Received:
    1,428
    Occupation:
    Hustler
    Location:
    Gurgaon
    Home Page:
    Hello,
    I am sure a lot of people already know this. I did not. I usually don't scrape much, but recently I had to scrape tons of keywords for one of my projects.

    I started out with ~30 private proxies and pretty soon all of them were banned on Google. I was running just one thread and 19 second delay. : /

    Long story short, I decided to scrape all search engines. I knew I would not get the best results but something is better than nothing. Plus I had a VPS just sitting there doing nothing so I had nothing to loose.

    To my surprise, even though my proxies were banned on Google, they have been successfully scraping using the "Google API" engine. Also search engines like startpage etc have very similar results to Google, I think they might even be using their APIs.

    I am currently running less than 100 threads.

    [​IMG]

    TL;DR: You can use "Google API" search engine in scrapebox with Google banned proxies too. The results might be a bit limited compared to the web version, but worth the effort methinks. Especially if you are scraping for tons of keywords like me.
     
  2. rayanJo

    rayanJo Junior Member

    Joined:
    Jan 26, 2016
    Messages:
    100
    Likes Received:
    6
    ya mate you r right this big probleam for user
     
  3. patomica

    patomica Regular Member

    Joined:
    Jul 27, 2011
    Messages:
    412
    Likes Received:
    97
    Is there a difference of the url's scraped on the google se on the api?. I'm testing the API now.
     
  4. Buzzika

    Buzzika Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2009
    Messages:
    1,161
    Likes Received:
    1,428
    Occupation:
    Hustler
    Location:
    Gurgaon
    Home Page:
    I don't think there is any difference. There are limitations however.
    Google API provides fewer results per query compared to their web version.
     
  5. patomica

    patomica Regular Member

    Joined:
    Jul 27, 2011
    Messages:
    412
    Likes Received:
    97
    I see, That's okay though as i'm just scraping 100 per footprint. Thanks for the tip :)
     
  6. 710fla

    710fla Jr. VIP Jr. VIP

    Joined:
    Aug 25, 2015
    Messages:
    748
    Likes Received:
    218
    You still running 19 second daily and using 30 private proxies?

    Have you tried this with public scraped proxies? I'm thinking of just scraping Google API and Startpage.

    And reason why you didn't get any results from the Startpage search engine?
     
    Last edited: Jun 9, 2016