1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[scrape request]

Discussion in 'Hire a Freelancer' started by bertalting, Dec 2, 2014.

  1. bertalting

    bertalting Newbie

    Joined:
    May 3, 2014
    Messages:
    18
    Likes Received:
    1
    Dear friends,

    I really need help to scrape a website

    transportation-services.findthebest.com

    1,4 million addresses from transport companies in the US are listed here with email addresses.
    When i use scraping bots i will be blocked within a few minutes.
    So i thought maybe when i set the visiting time to the website at 10 seconds (1 mail per 10 seconds) i also will be blocked.

    Do you guys know anything how to scrape this mf-er.


    If anyone can scrape this for me i will ofcourse offer some $$$

     
  2. bbrez1

    bbrez1 Power Member

    Joined:
    Feb 21, 2009
    Messages:
    675
    Likes Received:
    2,360
    I can code a custom scraper in about 2-3 days.
    If interested pm me with an offer.
     
  3. bertalting

    bertalting Newbie

    Joined:
    May 3, 2014
    Messages:
    18
    Likes Received:
    1

    There is one problem with that website. You will get blocked if you connection rate is too high.
    I tried it with 1 connection per 10 seconds. Afterwards i was blocked :(
     
  4. bbrez1

    bbrez1 Power Member

    Joined:
    Feb 21, 2009
    Messages:
    675
    Likes Received:
    2,360
    Proxies buddy, proxies :)
     
  5. bertalting

    bertalting Newbie

    Joined:
    May 3, 2014
    Messages:
    18
    Likes Received:
    1
    even those will get blocked :p
     
  6. prab1996

    prab1996 Elite Member

    Joined:
    Jan 8, 2013
    Messages:
    3,496
    Likes Received:
    2,028
    Occupation:
    your gf's <3 ♥♥♥♥
    Location:
    Prab1996.com
    Home Page:
    proxies ?
    -=-
     
  7. bertalting

    bertalting Newbie

    Joined:
    May 3, 2014
    Messages:
    18
    Likes Received:
    1
    then i need about 5000 proxies... anybody?
     
  8. MrBlue

    MrBlue Senior Member

    Joined:
    Dec 18, 2009
    Messages:
    950
    Likes Received:
    662
    Occupation:
    Web/Bot Developer
    Just sent you a PM.
     
  9. elhefe

    elhefe Newbie

    Joined:
    Jul 9, 2013
    Messages:
    41
    Likes Received:
    4
    I can do this. Catch me on skype: el.he.fe
     
  10. bbrez1

    bbrez1 Power Member

    Joined:
    Feb 21, 2009
    Messages:
    675
    Likes Received:
    2,360
    You can get 5000 public proxies in one day.