Need advice for setting up a big mobile proxy farm

Dongles.

With modules be ready to have headaches.

I mean they do work. But they are annoying sometimes to deal with. As @Bers666 already mentioned here.
Hahaha. Thank you for this. I feel like I am too stubborn and am about to make the same mistakes you have already made along the way. I guess I'm just better off listening to you. Anyway, for now I'll start by using dongles.

I am a little annoyed that I can't get better speeds on my dongles than 20-30 Mbit/s but I guess those are actually good enough for most applications.
 
https://ibb.co/L9GTbZ8
.

i connect my iPhone to the router that work as bridge to the proxy server.

But i dont know how tò configurate the OPNsense router.
In looking tò allows a router to act as an intermediary between a user and the PC (proxy server)

Other info About setup :

my proxy is only accessible from LAN

Pc running a Linux Ubuntu Os

the PC Is connected with a usb hub with One 4g sim dongles

With 3proxy i manage to create One 4g mobile proxy using sim and 4g data .

I have a Opsense router connected tò the Linux PC.

The client ( iphone) are connected via USB- ethernet cable tò the OS router

how tò configurate the OPNsense router?

Can you linka guide tò setup the router?

I Will use proxy for my own business .
I dont care if isn't scalable
 
Last edited:
@derregenrennt what will be your use case?
I have recently been getting back into Instagram botting. Last time I was involved in 2017.... oh how times have changed haha. I'm also doing a lot of webscraping currently for a gen ai project that a friend of mine and I are working on. As mentioned above my office internet is terrible and we are currently changing our network to 5g routers as that seems to be a faster and more reliable solution. That is how I stumbled across the fact that I can currently get incredibly cheap data plans (around 1/4th of normal cost). I don't think that this was intended by the network operator but I have been assured that it does work the way I imagine it so let's see. This led me to consider setting up my own 4g proxies because they are dirt cheap... so why not?
 
I have recently been getting back into Instagram botting. Last time I was involved in 2017.... oh how times have changed haha. I'm also doing a lot of webscraping currently for a gen ai project that a friend of mine and I are working on. As mentioned above my office internet is terrible and we are currently changing our network to 5g routers as that seems to be a faster and more reliable solution. That is how I stumbled across the fact that I can currently get incredibly cheap data plans (around 1/4th of normal cost). I don't think that this was intended by the network operator but I have been assured that it does work the way I imagine it so let's see. This led me to consider setting up my own 4g proxies because they are dirt cheap... so why not?
Do you Suggest any guide to setting up my own 4g mobile proxy ?
 
I have recently been getting back into Instagram botting. Last time I was involved in 2017.... oh how times have changed haha. I'm also doing a lot of webscraping currently for a gen ai project that a friend of mine and I are working on. As mentioned above my office internet is terrible and we are currently changing our network to 5g routers as that seems to be a faster and more reliable solution. That is how I stumbled across the fact that I can currently get incredibly cheap data plans (around 1/4th of normal cost). I don't think that this was intended by the network operator but I have been assured that it does work the way I imagine it so let's see. This led me to consider setting up my own 4g proxies because they are dirt cheap... so why not?
Instagram has changed a lot, especially recently.

Anyways, if you will run web automation, skip images/video files from being proxied through the browser.

You can use request interception in puppeteer or apply proxy based on request type, .e.g do not proxy requests that has file extensions of jpg,jpeg,gif,mp4,js etc etc etc

By doing this you will reduce your resource use and you will be more than fine even if your dongle speed will be slower than 5 Mbits.

It's the latency that is more important in this case, if you get above 200 it will be annoying to use.

This same approach can be done on phone but it will require some hacking and know-how.
 
Do you Suggest any guide to setting up my own 4g mobile proxy ?
For ease of use I have just started using proxysmart. It's well documented, cheap and easy to use. I can only recommend it so far.

Instagram has changed a lot, especially recently.

Anyways, if you will run web automation, skip images/video files from being proxied through the browser.

You can use request interception in puppeteer or apply proxy based on request type, .e.g do not proxy requests that has file extensions of jpg,jpeg,gif,mp4,js etc etc etc

By doing this you will reduce your resource use and you will be more than fine even if your dongle speed will be slower than 5 Mbits.

It's the latency that is more important in this case, if you get above 200 it will be annoying to use.

This same approach can be done on phone but it will require some hacking and know-how.
Thanks a bunch. Looks like I'm ready to give it a go :)

delete14.png
 
sorry it is outdated LTE cat4 equipment with double NAT. Yes for beginners and mid-advanced users it is OK to run farms on dongles ;-) For real PRO's - only LTE chips:)
What's the biggest farm you have run using LTE chips? as I don't like how unreliable they are. Right now my dongles don't require any maintenance at all. I couldn't reach this kind of reliability with LTE chips. btw, tried with proxysmart too, my dongles did work where proxysmart failed with lte chips. The failure was because of bad LTE signal in this case but anyways, I had all but problems with chips like they need to be hardreset etc. Perhaps I sourced them badly? Can you suggest where to source them?

well, some antifraud tests may be failed with this approach. They may ask you to load a 1x1 pixel image, and get a lot info from your request. If it is done not via proxy - the test is failed.
Yes, but it was about instagram's static resources, e.g. photos and videos on main feed or profiles.
 
It depends on many factors (power source, USB adaptors, USB hub, antennas, etc).
Tried everything,
- various power sources(powered hubs)
- different USB adaptors from different sources
- with or without antennas

I just couldn't make it work to be as stable as dongles are.

Technically, PPPS(per port power switch) hub could be an answer, there are some https://github.com/mvp/uhubctl available but I don't know if the hassle of changing everything gives the benefit as those old cat4 dongles are doing just fine.

And are reliable af.
 
what were they (model) ? what was the issue? (sudden reboots, going offline, loosing carrier)
1709573263366.png


And then couple of these

1709573636464.png


Mostly issue was with just losing the connection to PC(I tried many PCs), it was weird - like chip stopped being registered. And then there were like disconnects and losing carrier all the time but it was because bad signal I have where i live.

I thought about forcing specific bands but it didnt help too much. Never spent a lot of time on this tho, only couple days of testing was enough for me to get pissed off enough about it.

I'm not saying they doesnt work, obviously where good signal is available it's cool to push those superb speeds what was my main issue was that they required hard reset from time to time which was main deal breaker for me.

Perhaps I will spend time on more testing, if I could find reason why they were disconnecting all the time it would be awesome.



ok, but for the USA it is hard to find dongles, so LTE chips are very useful there.
Makes sense, I dont know much about US. have you tried zte wd580, these are what I will try in US market. I cant give any feedback yet tho.
 
What do you consider as stress test, what loads you get, what is considered as big load?
You're talking about a 200 mbits load for 500+ devices. For us, the usual load for 500 devices is no less than 5000 mbits. So, the regular load per device starts at 10 mbits or maybe around 15 mbits.

A "big load" we consider as CPU load > 70%. That is the our red flag where loading any further risks dropping the quality of the proxy connection. By the way, we calculate CPU load as 100% minus idle because the system CPU can get pretty high.

Question, why scraping is considered something big? If the client is smart and avoids loading static contents over proxied devices, it should not be a problem at all, no?
From what we have seen, scraping generates a ton of traffic. The difference between our experiences might be that we don't charge clients based on the volume of traffic they use. So, if a client has unlimited mobile data on their SIM, they can push through as much traffic as they want through the proxy.

you connect your phone proxies via backconnect right?
100%, yes. That's a key difference from your case.

I would also add, if you are allocating one device per physical core, then the impact of clock speed becomes almost negligible. But we live in a world where one core is shared among many processes. So, you end up with a "queue of proxies" on each core, and the higher the frequency, the faster they all get processed, minimizing the downtime for each process (each proxy connection).
 
@iproxyonline 10-15mbits per device is a heavy heavy use, sounds like streaming video or something. Or you just let one device be used by -n clients at same time.

Anyways, thank you for sharing details, incredibly interesting case you got.
 
10-15mbits per device is a heavy heavy use, sounds like streaming video or something. Or you just let one device be used by -n clients at same time.
Our clients could give access to their proxies to multiple users; it's their choice. For example, some arbitrage teams might use one phone to set up a couple of proxy accesses for their work. But those are more like exceptions. I think the answer lies in the fact that if you dont limit your clients traffic, they wont really limit themselves in using it lol :)

Or take, for example, one of our long-time clients who shared their experience scraping TikTok. A single one of his devices was pushing a load of 25 mbits. So, someone working through a proxy on social networks (to scrape or boting) will easily generate that kind of load.
Another client, with whom I was just chatting today, has a load of over 30 mbits. Though, this is a device using a 5G proxy, which significantly boosts the speed.
 
Back
Top
AdBlock Detected

We get it, advertisements are annoying!

Sure, ad-blocking software does a great job at blocking ads, but it also blocks useful features and essential functions on BlackHatWorld and other forums. These functions are unrelated to ads, such as internal links and images. For the best site experience please disable your AdBlocker.

I've Disabled AdBlock