1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Craigslist Scrapper and proxies needed

Discussion in 'Black Hat SEO Tools' started by JoeStu, Jun 22, 2017.

  1. JoeStu

    JoeStu Newbie

    Joined:
    Sep 18, 2015
    Messages:
    3
    Likes Received:
    0
    Anyone have much experience with scraping CL and the number of proxies etc needed?

    I have tried various proxy sites and they end up all being\getting banned and an looking into some services that say they offer specific CL pools, but I need some idea of how many to purchase. We have also had it suggested that we buy X amazon aws IPs (I think) and use them, but the setup cost etc without knowing if it would work seems high.

    Our use case is just grabbing ads, no posting, no responding, just the ads. So say ~420 cities => ~5000 ads a day.

    Our old system just grabbed all the RSS feeds, and ~15->20 IPs could do that but we are seeing that the RSS feeds are spotty at best on ads.

    So any input would be great. I have seen other topics here, and someone mentioned a CL forum, but I have not found it.
     
  2. MonteCarlos

    MonteCarlos Regular Member

    Joined:
    Aug 1, 2015
    Messages:
    310
    Likes Received:
    416
    Gender:
    Male
    Occupation:
    Wannabe Money-Maker
    Location:
    Lost in CyberSpace
    Try ArbitrageUnderdog, I don't think it needs proxies. Craigslist talking is forbidden in this forum BTW...Not without a reason though...
     
  3. JoeStu

    JoeStu Newbie

    Joined:
    Sep 18, 2015
    Messages:
    3
    Likes Received:
    0
    Thanks. I will check the tool out, but gut says not quite right as I need the data a little more raw then in their tool.

    I wasn't aware of the rule for this forum ( I am not even sure where the rules are listed.) Is there a better place\forum for them? You can PM if needed.
     
  4. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    20,081
    Likes Received:
    10,408
    Gender:
    Male
    Occupation:
    PROXIES .
    Location:
    UK - ALWAYS ON BHW
    Home Page:
    always been a bhw policy that c-list chat is a no go.
    so just be careful mate
     
  5. Javardo69

    Javardo69 Junior Member

    Joined:
    Jul 19, 2014
    Messages:
    106
    Likes Received:
    6
    scraping ads its easy, but if you want to scrape the contact details thats more complex (need captcha service) and you get your ip's banned after a while so you have to constantly scrape non banned ips.
     
  6. headon

    headon Newbie

    Joined:
    Jul 2, 2017
    Messages:
    1
    Likes Received:
    0
    You can develop scraper with ubot or BrowserAutomationStudio. proxies + captcha service +
     
  7. proxygo

    proxygo Jr. VIP Jr. VIP

    Joined:
    Nov 2, 2008
    Messages:
    20,081
    Likes Received:
    10,408
    Gender:
    Male
    Occupation:
    PROXIES .
    Location:
    UK - ALWAYS ON BHW
    Home Page:
    i will repeat craigs list stuff not allowed on bhw
     
  8. Elmokhtatif

    Elmokhtatif Newbie

    Joined:
    Jul 3, 2017
    Messages:
    1
    Likes Received:
    0
    Gender:
    Male
    thanks you pro