1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

hrefer scraping 24/7

Discussion in 'Proxies' started by lanbo, Aug 9, 2011.

  1. lanbo

    lanbo Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 23, 2009
    Messages:
    3,435
    Likes Received:
    595
    Home Page:
    Anyone how many proxies I'd need to harvest links 24/7?
    What settings should I put them at? (ie how many threads?) so they don't get banned from google?

    Looking to scrape big lists every day
     
  2. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,260
    Likes Received:
    8,706
    all proxies get banned from google after so many scrapes
    for upto 3 hrs. thats the way it is
     
  3. lanbo

    lanbo Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 23, 2009
    Messages:
    3,435
    Likes Received:
    595
    Home Page:
    hmm so how many would i need to scrape 24/7?
     
  4. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,260
    Likes Received:
    8,706
    hard to say i no with scrapebox when you scrape say
    100000 keywords proxies get a 3hr block
    so work it out
     
  5. lanbo

    lanbo Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 23, 2009
    Messages:
    3,435
    Likes Received:
    595
    Home Page:
    how many proxies are being used for that?
     
  6. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,260
    Likes Received:
    8,706
    3 private i got that from some 1 using my service here
    ille post u his comment link only the bit about 3 proxies
    and keyword scraping applys ignore the rest
    read the 2nd - 4th line about private proxies
    ignore the rest

    this part

    Just do a little test and take 100000 keywords, use 3 proxies and start harvesting.
    After a while SB will come to a grinding halt. Google blocked your proxies...
    Eventually they will clear again but all this time you can't continue with your project.

    http://www.blackhatworld.com/blackhat-seo/2575142-post768.html