JohnnyOmm
Registered Member
- Oct 12, 2021
- 71
- 9
what other stuff? post it
Need advice for setting up a big mobile proxy farm
- Thread starter _february
- Start date
Natael
Newbie
- Apr 28, 2024
- 34
- 4
@lucky.sparks mentioned (long time ago) using 2.0 usb hub instead of 3.0. Is it still a valid choice?
There is not many manufacturers offering 2.0 usb hub, I mean quality ones, not cheap useless stuff from cheap websites.
There is not many manufacturers offering 2.0 usb hub, I mean quality ones, not cheap useless stuff from cheap websites.
- Oct 6, 2020
- 2,626
- 9,931
It depends on dongles you use, if you use 2.0 dongles, there is no point of using 3.0 hubs.
There are more than couple good manufacturers for 2.0 hubs, best is sipolar and then orico. They are very good quality and are cheap if you buy from alibaba, not from resellers in amazon.
Natael
Newbie
- Apr 28, 2024
- 34
- 4
Thanks.
Many of you mentioned ZTE MF79U (usb 2.0).
Does ZTE MF79N also a good choice (older version but cheaper)?
- Oct 6, 2020
- 2,626
- 9,931
You can get MF79U as low as 19$ per dongle on alibaba.
MF79N looks almost same so I expect it to be as good as U version. I haven't tested it.
Buy both and compare.
JohnnyOmm
Registered Member
- Oct 12, 2021
- 71
- 9
why bring up the hardware if youre not even gonna talk about it and then complain that you arent making sales?
concerned as in health defects? good question
- Jul 8, 2021
- 293
- 209
Sierra wireless, Quectel
Magic Networks
Junior Member
- May 28, 2024
- 174
- 64
I was using a 50-dongle farm (huawei 3372h) for more than a year.
Dongles are quite stable and 3372h and MF79U have TTL and IMEI fixes (so you can use cheap smartphone tarrifs)
I was using a little bit different solution than you all have. With no local server and USB-hubs.
Every dongle was connected to a cheap small router which was handling a VPN connection to the remote server (the actual proxy server with 3proxy)
This solution is not cheaper but if you count the price of the local server with good usb hubs and fast connection - the price will be comparable.
The main advantage - that you don't need an expensive local server with static IP and high-speed optic fiber channel. Actually there is no fiber optics in many locations.
Later I had to switch to 5g networks and now I am using smartphones. There are several stable android applications that work fine, but I had to develop my own.
I would be grateful if anyone can advise any affordable 5g dongles with ability to change IMEI and/or TTL.
- Jul 8, 2021
- 293
- 209
do they ever exist? "dongles" stopped being manufactured around 2018. Future 5G-capable mobile equipment are either Phones or MiFi routers (also almost ended) or 4G/5G-LAN routers.
Magic Networks
Junior Member
- May 28, 2024
- 174
- 64
Exactly, That's why I ask.
But I know that there are some.
Maybe they don't look like classic sticks(they are bigger) but they are still USB devices based on SIM8202G or QUECTEL RG200U.
Some of them based on raspberry or somethig like that. But they are quite expensive.
Please let me know if you know any solutions.
- Jul 8, 2021
- 293
- 209
Rather, you believe they exist. 4gltemall website - apply filters - find 5g dongles. Share with us what you found, thanks.
noellarkin
Senior Member
- Mar 14, 2021
- 1,006
- 1,492
am curious about this, too
Magic Networks
Junior Member
- May 28, 2024
- 174
- 64
Your help is invaluable ) Thank you bro.
Just a couple of examples:
https://www.alibaba.com/product-det...html?spm=a2700.7735675.0.0.502eaQnmaQnmhf&s=p
https://5gmodemrouter.com/product/5g-dongle/
- Aug 7, 2019
- 519
- 169
Besides hardware, are there any software knowledge requirements or any guides you would recommend or YouTube channels, for someone who wants to get into the business?Sure. I would recommend sticking with the idea of having more small clusters of mini PC's or SBC's than one large one just because of fault tolerance. But this depends on whether you will be around them or not and how good uptime you want to reach. If only for your botting setup and you plan to be around it so you can do maintenance I would recommend going with a good tower PC.
Right now my tower specs are:
- Intel, Core i5-12500
- Thermaltake, Core V1 Snow Edition / Black edition. Because it looks dope.
- motherboard Gigabyte, H610I
- cheapest ddr4 RAM and cheapest/smallest SSD
With these specs it shows 3k connections open to this box
# ss -o state established | wc -l
3044
and it's practically sleeping in terms of CPU load.
View attachment 324463
This is what it looks like:
So you really dont need a powerful box to run 100 dongles on a single PC.
For hubs just go with orico or sipolar USB hubs.
Don't do experiments with custom hubs as I did:
This is BAD. While this hub is very cool because it works from a regular PSU, has a lot of slots with good spacing between for good airflow, this is where I learned the hard truth about what USB controllers are and how they work. With this setup on medium to high load speed per dongle decreased to glorious 1 Mbit/s and latency went up to 1000ms.
Now I'm working to switch setup on boards like these which I haven't seen others using.
While they still have 1.5A and 5V output to support dongle on high load, there is only 16 2.0 ports, with good spacing for ideal airflow and double power slots for fault tolerance.
For Mikrotik I switched to RB4011iGS+5HacQ2HnD-IN. I kinda don't like it has passive cooling but specs/price ratio is very good and it has built in wireless which makes things easier when you want to quickly go to site, do some maintenance and be on your way. Before I used more regular mikrotik router boards.
Right now I have around 1000 modems hosted in multiple countries,
I have tried it all, this is my approx history of upgrades:
- started with a single PC, Orico hubs and Huawei modems
- changed standalone PC setup for XU4
- started using mikrotik's router boards
- changed Huawei Dongles for ZTE(lifechanging experience)
- experimented with $1000 worth of various hubs to find what works best
- for reasons unclear even to myself I changed some XU4 to a single PC approach again hosting 80-100 dongles per PC
- only couple power outages and hardware faults did show that single PC approach is bad for high uptime requirements
- now I'm switching back to SBC approach and using vim4 from Khadas, it's like XU4 on steroids.
Because commonly advised is hobbyist experiment on a proof of concept level, it will work but you wont be able to run more than 4-6 dongles per rpi. I dont know why, never had interest to investigate why it's so, either it comes from rpi chipset limits or software, i dont know, dont care, XU4 is cheaper and way more powerful. That said XU4 is soon to be discontinued, so will need to look for other options.
- Both are more cost effective as I get my ZTE for less than $20 per dongle with pretty much unlimited supply.
- ZTE has never frozen on me, Huawei had tendency to freeze and stop responding and only hard reset did help.
- ZTE vs Huawei i got better speeds for some carriers. I never experimented much with it but i did notice about 20% difference.
- XU4 uses eMMC where rpi uses microSD. Yes yes, you can probably upgrade rpi with all sorts of stuff to support SSD etc, don't care.
What's significant traffic? Why you want to run big traffic, you need to skip any static resources in your bot from being proxied, it will decrease the use a lot. Unless you plan to use real android phones, then it's another story, and a deep one too haha. But even there optimizations are possible to handle big loads properly.
And here starts the rabbit hole,
USB Controller chipset endpoint limits.
I will try to simplify as much as I can.
- Every USB chipset has limits on how many USB devices can be connected to it.
- With USB 2.0(white color) you will be able to connect 20-40 devices, with USB 3.0(blue color) you will be able to connect only 16 devices
- Max supported device count comes from how many endpoints USB device uses, technically USB 2.0 supports 127 devices but you will have bad time trying to do so
- ZTE/Huawei dongles are USB 2.0, so it doesn't matter if you use USB 3.0 hubs or 3.0 ports on your m/b, it will go backward compatibility to USB 2.0
- USB 2.0 is half-duplex, which means data travels one way at a time, which means it's slow. If you add more than -N devices to the USB controller, it will become slower with each additional device if they are all on high load.
So to run on PC you will need good USB PCIe card with separate channels per port to get most of your setup.
All the above is only about hardware. You will need software to manage it all too. For inhouse use simple node.js script that generates 3proxy config will be enough.
My coffee got cold writing this. Will come back later if I will remember to add something here.
Would you need a background in IT? If so could you point me in the right direction to educate myself
I just ordered 1 odroid xu4, 2 mf79u, and a 4 port USB hub to familiarize myself on a small scale. A mini project.
I also have a small army of mobile phones (from an older project) but not exactly sure how I want to use them in regards to selling proxies. (I did create some android proxies using some GitHub Mobile app to get a feel for it)
Last edited:
ImBackBaby
Junior Member
- May 23, 2023
- 104
- 97
Hey nice to see someone else at roughly the same point as me. I also have 1 proxy running on a phone and was playing around with a Raspberry Pi and Huawei Stick before.
But now decided to order 1 Odroid XU4 and 6mf79u (got 2 Sim cards right now, but should be easy to get another 4 I believe), I already have a USB hub with it's own power supply.
Right now I'm pretty busy with my day-job but will probably work on setting some stuff up whenever I have time. If you want to connect feel free to message me @The0ry and maybe we can solve some issues that come up together.
Somewhere else I saw @lucky.sparks and other recommend Proxysmart and I gotta say this looks like a really promising project. Personally I will also look into building a whitelist though to at least somewhat cover my ass when it comes to legal aspects
did you switch back to a pc from XU4 because proxysmart require 64 bits OS ? or you are using your own software to manage your proxies.Sure. I would recommend sticking with the idea of having more small clusters of mini PC's or SBC's than one large one just because of fault tolerance. But this depends on whether you will be around them or not and how good uptime you want to reach. If only for your botting setup and you plan to be around it so you can do maintenance I would recommend going with a good tower PC.
Right now my tower specs are:
- Intel, Core i5-12500
- Thermaltake, Core V1 Snow Edition / Black edition. Because it looks dope.
- motherboard Gigabyte, H610I
- cheapest ddr4 RAM and cheapest/smallest SSD
With these specs it shows 3k connections open to this box
# ss -o state established | wc -l
3044
and it's practically sleeping in terms of CPU load.
View attachment 324463
This is what it looks like:
So you really dont need a powerful box to run 100 dongles on a single PC.
For hubs just go with orico or sipolar USB hubs.
Don't do experiments with custom hubs as I did:
This is BAD. While this hub is very cool because it works from a regular PSU, has a lot of slots with good spacing between for good airflow, this is where I learned the hard truth about what USB controllers are and how they work. With this setup on medium to high load speed per dongle decreased to glorious 1 Mbit/s and latency went up to 1000ms.
Now I'm working to switch setup on boards like these which I haven't seen others using.
While they still have 1.5A and 5V output to support dongle on high load, there is only 16 2.0 ports, with good spacing for ideal airflow and double power slots for fault tolerance.
For Mikrotik I switched to RB4011iGS+5HacQ2HnD-IN. I kinda don't like it has passive cooling but specs/price ratio is very good and it has built in wireless which makes things easier when you want to quickly go to site, do some maintenance and be on your way. Before I used more regular mikrotik router boards.
Right now I have around 1000 modems hosted in multiple countries,
I have tried it all, this is my approx history of upgrades:
- started with a single PC, Orico hubs and Huawei modems
- changed standalone PC setup for XU4
- started using mikrotik's router boards
- changed Huawei Dongles for ZTE(lifechanging experience)
- experimented with $1000 worth of various hubs to find what works best
- for reasons unclear even to myself I changed some XU4 to a single PC approach again hosting 80-100 dongles per PC
- only couple power outages and hardware faults did show that single PC approach is bad for high uptime requirements
- now I'm switching back to SBC approach and using vim4 from Khadas, it's like XU4 on steroids.
Because commonly advised is hobbyist experiment on a proof of concept level, it will work but you wont be able to run more than 4-6 dongles per rpi. I dont know why, never had interest to investigate why it's so, either it comes from rpi chipset limits or software, i dont know, dont care, XU4 is cheaper and way more powerful. That said XU4 is soon to be discontinued, so will need to look for other options.
- Both are more cost effective as I get my ZTE for less than $20 per dongle with pretty much unlimited supply.
- ZTE has never frozen on me, Huawei had tendency to freeze and stop responding and only hard reset did help.
- ZTE vs Huawei i got better speeds for some carriers. I never experimented much with it but i did notice about 20% difference.
- XU4 uses eMMC where rpi uses microSD. Yes yes, you can probably upgrade rpi with all sorts of stuff to support SSD etc, don't care.
What's significant traffic? Why you want to run big traffic, you need to skip any static resources in your bot from being proxied, it will decrease the use a lot. Unless you plan to use real android phones, then it's another story, and a deep one too haha. But even there optimizations are possible to handle big loads properly.
And here starts the rabbit hole,
USB Controller chipset endpoint limits.
I will try to simplify as much as I can.
- Every USB chipset has limits on how many USB devices can be connected to it.
- With USB 2.0(white color) you will be able to connect 20-40 devices, with USB 3.0(blue color) you will be able to connect only 16 devices
- Max supported device count comes from how many endpoints USB device uses, technically USB 2.0 supports 127 devices but you will have bad time trying to do so
- ZTE/Huawei dongles are USB 2.0, so it doesn't matter if you use USB 3.0 hubs or 3.0 ports on your m/b, it will go backward compatibility to USB 2.0
- USB 2.0 is half-duplex, which means data travels one way at a time, which means it's slow. If you add more than -N devices to the USB controller, it will become slower with each additional device if they are all on high load.
So to run on PC you will need good USB PCIe card with separate channels per port to get most of your setup.
All the above is only about hardware. You will need software to manage it all too. For inhouse use simple node.js script that generates 3proxy config will be enough.
My coffee got cold writing this. Will come back later if I will remember to add something here.
tried to run proxysmart on ubuntu with XU4 but failed to install.
- Oct 6, 2020
- 2,626
- 9,931
I do not use proxysmart but node.js scripts to manage everything
"Now I'm working to switch setup on boards like these which I haven't seen others using."Sure. I would recommend sticking with the idea of having more small clusters of mini PC's or SBC's than one large one just because of fault tolerance. But this depends on whether you will be around them or not and how good uptime you want to reach. If only for your botting setup and you plan to be around it so you can do maintenance I would recommend going with a good tower PC.
Right now my tower specs are:
- Intel, Core i5-12500
- Thermaltake, Core V1 Snow Edition / Black edition. Because it looks dope.
- motherboard Gigabyte, H610I
- cheapest ddr4 RAM and cheapest/smallest SSD
With these specs it shows 3k connections open to this box
# ss -o state established | wc -l
3044
and it's practically sleeping in terms of CPU load.
View attachment 324463
This is what it looks like:
So you really dont need a powerful box to run 100 dongles on a single PC.
For hubs just go with orico or sipolar USB hubs.
Don't do experiments with custom hubs as I did:
This is BAD. While this hub is very cool because it works from a regular PSU, has a lot of slots with good spacing between for good airflow, this is where I learned the hard truth about what USB controllers are and how they work. With this setup on medium to high load speed per dongle decreased to glorious 1 Mbit/s and latency went up to 1000ms.
Now I'm working to switch setup on boards like these which I haven't seen others using.
While they still have 1.5A and 5V output to support dongle on high load, there is only 16 2.0 ports, with good spacing for ideal airflow and double power slots for fault tolerance.
For Mikrotik I switched to RB4011iGS+5HacQ2HnD-IN. I kinda don't like it has passive cooling but specs/price ratio is very good and it has built in wireless which makes things easier when you want to quickly go to site, do some maintenance and be on your way. Before I used more regular mikrotik router boards.
Right now I have around 1000 modems hosted in multiple countries,
I have tried it all, this is my approx history of upgrades:
- started with a single PC, Orico hubs and Huawei modems
- changed standalone PC setup for XU4
- started using mikrotik's router boards
- changed Huawei Dongles for ZTE(lifechanging experience)
- experimented with $1000 worth of various hubs to find what works best
- for reasons unclear even to myself I changed some XU4 to a single PC approach again hosting 80-100 dongles per PC
- only couple power outages and hardware faults did show that single PC approach is bad for high uptime requirements
- now I'm switching back to SBC approach and using vim4 from Khadas, it's like XU4 on steroids.
Because commonly advised is hobbyist experiment on a proof of concept level, it will work but you wont be able to run more than 4-6 dongles per rpi. I dont know why, never had interest to investigate why it's so, either it comes from rpi chipset limits or software, i dont know, dont care, XU4 is cheaper and way more powerful. That said XU4 is soon to be discontinued, so will need to look for other options.
- Both are more cost effective as I get my ZTE for less than $20 per dongle with pretty much unlimited supply.
- ZTE has never frozen on me, Huawei had tendency to freeze and stop responding and only hard reset did help.
- ZTE vs Huawei i got better speeds for some carriers. I never experimented much with it but i did notice about 20% difference.
- XU4 uses eMMC where rpi uses microSD. Yes yes, you can probably upgrade rpi with all sorts of stuff to support SSD etc, don't care.
What's significant traffic? Why you want to run big traffic, you need to skip any static resources in your bot from being proxied, it will decrease the use a lot. Unless you plan to use real android phones, then it's another story, and a deep one too haha. But even there optimizations are possible to handle big loads properly.
And here starts the rabbit hole,
USB Controller chipset endpoint limits.
I will try to simplify as much as I can.
- Every USB chipset has limits on how many USB devices can be connected to it.
- With USB 2.0(white color) you will be able to connect 20-40 devices, with USB 3.0(blue color) you will be able to connect only 16 devices
- Max supported device count comes from how many endpoints USB device uses, technically USB 2.0 supports 127 devices but you will have bad time trying to do so
- ZTE/Huawei dongles are USB 2.0, so it doesn't matter if you use USB 3.0 hubs or 3.0 ports on your m/b, it will go backward compatibility to USB 2.0
- USB 2.0 is half-duplex, which means data travels one way at a time, which means it's slow. If you add more than -N devices to the USB controller, it will become slower with each additional device if they are all on high load.
So to run on PC you will need good USB PCIe card with separate channels per port to get most of your setup.
All the above is only about hardware. You will need software to manage it all too. For inhouse use simple node.js script that generates 3proxy config will be enough.
My coffee got cold writing this. Will come back later if I will remember to add something here.
which one is this ? which provide good spacing and double power slots.