what proxies to go for?

P

proxybros

Newbie
Joined
Apr 7, 2024
Messages
17
Reaction score
5
i'm scraping a cloudflare protected site. I'm making about 20 requests every 3-4 seconds. Previously I used a ipv6 provider from here and that worked perfect and cheap for 4 months. The site then caught on and those get blocked very quickly.

I'm considering trying static residential, about 4 or 5 and rotating them in my code. Rotating residential would probably be too costly. Is this likely to work good for my use case? I also rotate user agent on each request.

Any suggestions on providers? Thanks
 
Advertise on BHW
Your approach can, to a certain extent, increase the diversity of requests and reduce risks, but the cost is also relatively high. You might also give dynamic residential IPs a try. It is recommended that you find a proxy provider with rich IP resources, high stability, and customized services, and customize your IP pool according to your needs.
 
Thordata said:
Your approach can, to a certain extent, increase the diversity of requests and reduce risks, but the cost is also relatively high. You might also give dynamic residential IPs a try. It is recommended that you find a proxy provider with rich IP resources, high stability, and customized services, and customize your IP pool according to your needs.
Click to expand...
How will it be high? At most, ISP proxies are about $3-5 each and I’ll buy 5 so about $15-25 a month.

I’m not sure what dynamic residential is. Is that rotating residential? If so those charge for every GB of traffic and with the number of requests I’m making it seems the costs would be astronomical.
 
Depends how many requests you're making. If you're spacing them out then a few static ISP proxies could work
 
Proxy Cake said:
Depends how many requests you're making. If you're spacing them out then a few static ISP proxies could work
Click to expand...
I basically have 20 or so URLs and I’m sending requests asynchronously every 3 seconds. So if I get 4-5 proxies then 4 requests per proxy.
 
proxybros said:
How will it be high? At most, ISP proxies are about $3-5 each and I’ll buy 5 so about $15-25 a month.

I’m not sure what dynamic residential is. Is that rotating residential? If so those charge for every GB of traffic and with the number of requests I’m making it seems the costs would be astronomical.
Click to expand...
yes,You can give it a try.
 
Nowadays mobiel rpoxies are ideal for such cases. They can be used to avoid blocks and are dynamic
 
YellowBush said:
I would try rotating residential IPs and/or mobile.
Click to expand...
I need to calculate how much GB i’d need but I’m pretty sure it works out too much for my use case.

I’m planning on testing static residential/ISP proxies and rotating them in my code. Currently looking for a reliable provider
 
Yes, static residential proxies could work well for your use case if you keep the request rate reasonable and maintain good rotation with user agents and headers. Consider providers like Bright Data, Smartproxy, or Soax, as they offer reliable static residential proxies.
 
proxybros said:
i'm scraping a cloudflare protected site. I'm making about 20 requests every 3-4 seconds. Previously I used a ipv6 provider from here and that worked perfect and cheap for 4 months. The site then caught on and those get blocked very quickly.

I'm considering trying static residential, about 4 or 5 and rotating them in my code. Rotating residential would probably be too costly. Is this likely to work good for my use case? I also rotate user agent on each request.

Any suggestions on providers? Thanks
Click to expand...
Do you have any requirements for the duration of each IP, or is it acceptable for the IPs to rotate as long as they remain online?
 
proxybros said:
i'm scraping a cloudflare protected site. I'm making about 20 requests every 3-4 seconds. Previously I used a ipv6 provider from here and that worked perfect and cheap for 4 months. The site then caught on and those get blocked very quickly.

I'm considering trying static residential, about 4 or 5 and rotating them in my code. Rotating residential would probably be too costly. Is this likely to work good for my use case? I also rotate user agent on each request.

Any suggestions on providers? Thanks
Click to expand...
bhw has plenty of providers
https://www.blackhatworld.com/forums/proxies-for-sale.112/
 
proxybros said:
Any suggestions on providers? Thanks
Click to expand...
Your strategy of trying static residential proxies and rotating them, along with user agents, is a good start. However:
  • Static residential: Can work well if IPs aren't flagged; ensure they are high-quality and not reused by others.
  • Rotating residential: Although costlier, it offers more reliability if the site aggressively blocks static IPs.
    Suggestions:
  • Consider providers: Oxylabs, Bright Data, or Soax for residential proxies.
  • Additions: Use a CAPTCHA-solving service and implement randomized request intervals to mimic human behavior.
    Test with a small batch before scaling up.
 
There are some mobile proxy providers which give residential IPs plus offer IP rotation through API if that sort of thing is up your alley.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top