1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scraping, how much is to much.

Discussion in 'Proxies' started by mward, Oct 24, 2010.

  1. mward

    mward Newbie

    Joined:
    Oct 19, 2010
    Messages:
    43
    Likes Received:
    3
    Occupation:
    manager
    Location:
    uk
    Simple question, what would you say is a sensible amount of scrapes of a site per hour or min? If you didn't want to get banned or found out.
    Posted via Mobile Device
     
  2. cjw10

    cjw10 Junior Member

    Joined:
    Jan 7, 2010
    Messages:
    116
    Likes Received:
    32
    Occupation:
    Student
    Location:
    Illinois
    Use proxies and it doesn't matter if your banned.
     
  3. bhprince

    bhprince Registered Member

    Joined:
    Oct 19, 2010
    Messages:
    57
    Likes Received:
    8
    you can get either private proxies or just use scapebox to find some public ones.
     
  4. CyrusVirus

    CyrusVirus BANNED BANNED Premium Member

    Joined:
    Aug 20, 2009
    Messages:
    1,110
    Likes Received:
    686
    lol wierd that you posted this in the proxies section, are you asking how much is too much to scrape for private proxies?
    or just in general without proxies at all.

    if you have just 1 site, then 2 proxies should do it.
    in general, throw in a few free proxies and just scrape.
     
  5. mward

    mward Newbie

    Joined:
    Oct 19, 2010
    Messages:
    43
    Likes Received:
    3
    Occupation:
    manager
    Location:
    uk
    Sorry my fault, I have private proxies but still feel I should limit the amount of request as if the proxies get banned I will have to buy more
    Posted via Mobile Device
     
  6. Ardit

    Ardit Junior Member

    Joined:
    Oct 14, 2009
    Messages:
    119
    Likes Received:
    18
    Occupation:
    Marketing Expert
    Location:
    Facebook, Twitter, Google+ & Tumblr
    I think it depends on how old your domain is. If it's a new domain you'll get sandboxed very fast.
     
  7. mward

    mward Newbie

    Joined:
    Oct 19, 2010
    Messages:
    43
    Likes Received:
    3
    Occupation:
    manager
    Location:
    uk
    My domain? 10 years
    Posted via Mobile Device
     
  8. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,294
    Likes Received:
    8,714
    thats why the golden rule is

    use public proxies for scraping
    private ones for posting
    thought most people new that
     
  9. mward

    mward Newbie

    Joined:
    Oct 19, 2010
    Messages:
    43
    Likes Received:
    3
    Occupation:
    manager
    Location:
    uk
    public ones are fair to unreliable for that we need, it need to work everytime, no just on an off, we tested it with public ones and it failed, but thanks anyway