What proxy service are you using for scrapping?

G

GonzaloS

Newbie
Joined
Aug 17, 2017
Messages
21
Reaction score
5
Hi,

I was using storm proxies but it doesn't seem to be working at the moment.
Any recommendation for a rotating proxy service with no bandwidth limit and under $50/month (up to 10 threads)?

Thanks
 
Advertise on BHW
What you scraping?
Also, threads = connections = every http request = connection= every js,html,css files is a thread.
 
A classifies website that stops my connection every time I make more than 500 requests/day, I am also only requesting the main page, I don't request js, css, etc.
 
GonzaloS said:
I am also only requesting the main page, I don't request js, css, etc.
Click to expand...


Do you use a plain scraper (just requests and downloads) or a browser emulator (like Selenium)?

Also, did you try changing/rotating requests headers?
 
Actually, a lot can depend on the target you're trying to scrape. Some providers have pools that are not blocked on Amazon, for example, while some are better for Google.
 
GonzaloS said:
Hi,

I was using storm proxies but it doesn't seem to be working at the moment.
Any recommendation for a rotating proxy service with no bandwidth limit and under $50/month (up to 10 threads)?

Thanks
Click to expand...


why not you use private proxies for scraping ??
 
What my research of the proxy market tells me, I don't think you'll get good quality proxies that too with rotating proxy service under $50/month.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top