Craigslist Scrapper and proxies needed

JoeStu

Newbie
Joined
Sep 18, 2015
Messages
4
Reaction score
0
Anyone have much experience with scraping CL and the number of proxies etc needed?

I have tried various proxy sites and they end up all being\getting banned and an looking into some services that say they offer specific CL pools, but I need some idea of how many to purchase. We have also had it suggested that we buy X amazon aws IPs (I think) and use them, but the setup cost etc without knowing if it would work seems high.

Our use case is just grabbing ads, no posting, no responding, just the ads. So say ~420 cities => ~5000 ads a day.

Our old system just grabbed all the RSS feeds, and ~15->20 IPs could do that but we are seeing that the RSS feeds are spotty at best on ads.

So any input would be great. I have seen other topics here, and someone mentioned a CL forum, but I have not found it.
 
Try ArbitrageUnderdog, I don't think it needs proxies. Craigslist talking is forbidden in this forum BTW...Not without a reason though...
 
Thanks. I will check the tool out, but gut says not quite right as I need the data a little more raw then in their tool.

I wasn't aware of the rule for this forum ( I am not even sure where the rules are listed.) Is there a better place\forum for them? You can PM if needed.
 
scraping ads its easy, but if you want to scrape the contact details thats more complex (need captcha service) and you get your ip's banned after a while so you have to constantly scrape non banned ips.
 
You can develop scraper with ubot or BrowserAutomationStudio. proxies + captcha service +
 
i will repeat craigs list stuff not allowed on bhw
 
Back
Top