1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

hrefer scraping 24/7

Discussion in 'Proxies' started by lanbo, Aug 9, 2011.

  1. lanbo

    lanbo Jr. VIP Jr. VIP

    Joined:
    Aug 23, 2009
    Messages:
    3,590
    Likes Received:
    614
    Home Page:
    Anyone how many proxies I'd need to harvest links 24/7?
    What settings should I put them at? (ie how many threads?) so they don't get banned from google?

    Looking to scrape big lists every day
     
  2. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    18,379
    Likes Received:
    10,059
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    all proxies get banned from google after so many scrapes
    for upto 3 hrs. thats the way it is
     
  3. lanbo

    lanbo Jr. VIP Jr. VIP

    Joined:
    Aug 23, 2009
    Messages:
    3,590
    Likes Received:
    614
    Home Page:
    hmm so how many would i need to scrape 24/7?
     
  4. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    18,379
    Likes Received:
    10,059
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    hard to say i no with scrapebox when you scrape say
    100000 keywords proxies get a 3hr block
    so work it out
     
  5. lanbo

    lanbo Jr. VIP Jr. VIP

    Joined:
    Aug 23, 2009
    Messages:
    3,590
    Likes Received:
    614
    Home Page:
    how many proxies are being used for that?
     
  6. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    18,379
    Likes Received:
    10,059
    Occupation:
    PROVIDING PROXIES FOR GSA SCRAPING.
    Location:
    BHW
    Home Page:
    3 private i got that from some 1 using my service here
    ille post u his comment link only the bit about 3 proxies
    and keyword scraping applys ignore the rest
    read the 2nd - 4th line about private proxies
    ignore the rest

    this part

    Just do a little test and take 100000 keywords, use 3 proxies and start harvesting.
    After a while SB will come to a grinding halt. Google blocked your proxies...
    Eventually they will clear again but all this time you can't continue with your project.

    http://www.blackhatworld.com/blackhat-seo/2575142-post768.html