1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Have I successfully bypassed Google's scraping detection?

Discussion in 'Proxies' started by drerock, Oct 21, 2011.

  1. drerock

    drerock Newbie

    Joined:
    Oct 21, 2011
    Messages:
    2
    Likes Received:
    0
    I've been scraping Google using 20 proxies for about 4 days straight taking precautions not too use each proxy too much or too often. Normally, if I use a bad proxy, I get blocked within the first 30 minutes. Am I in the clear now or does Google have multiple layers of detection, some of which don't kick in until days later?
     
  2. NIXMY

    NIXMY Regular Member Premium Member

    Joined:
    Sep 26, 2010
    Messages:
    481
    Likes Received:
    321
    Location:
    myproxylists.com
    Home Page:
    I don't know exact amounts how many requests a single IP can do before it will be temp. banned. How many requests you have done per IP approx.? You can try it simply by using one IP and see many requests you can do before it will get blocked.

    I need to query google for my own purposes but im using about 100 clean proxy randomly, haven't experienced yet issues.

    PS. I don't think so you have bypassed anything, you just did not scraped too much.
     
  3. ricardo_bresil

    ricardo_bresil Newbie

    Joined:
    Jul 29, 2009
    Messages:
    19
    Likes Received:
    7
    Hello,

    Some info about scraping :

    Code:
    searchnewscentral_com/20110928186/General-SEO/how-to-scrape-search-engines-without-pissing-them-off.html
    Remove _
     
    • Thanks Thanks x 1
  4. drerock

    drerock Newbie

    Joined:
    Oct 21, 2011
    Messages:
    2
    Likes Received:
    0
    ricardo_bresil: Thanks. Interesting. I hadn't read that but it is almost exactly what I'm doing except that the minimum wait time is a little less for me. I will save the article nevertheless.

    NIXMY: The algorithm I use allows each proxy to act independently. It doesn't matter if I have 1 or 100, each individual proxy searches differently and at different times. I've tried the test with 1 proxy and with 20 proxies and both have worked well. The only way that I can see a proxy being affected by other proxies is if they are in the same subnet. I have 20 proxies and five different subnets. It seems that is enough subnets...for now. BTW I'm not allowing each proxy to be used more than 500 times in 24 hours and I'm waiting at least 15 seconds before a proxy is used again.

    So the questions still remains: I am in the clear after 4 days of not being blocked?
     
  5. kokoloko75

    kokoloko75 Elite Member

    Joined:
    Jan 1, 2011
    Messages:
    1,628
    Likes Received:
    1,936
    Occupation:
    Design director
    Location:
    Paris (France)
    Just be human.

    Beny
     
  6. richboy121

    richboy121 Newbie

    Joined:
    Oct 8, 2011
    Messages:
    16
    Likes Received:
    1
    :)
    haaaaaaaahha what a saying:):)