Badboy Enterprise
BANNED
- Jan 24, 2015
- 1,189
- 316
Thanks man
[Q] How do you build your own 4g (mobile) proxy network?
Heisenberg's Hat
Regular Member
- Apr 5, 2019
- 227
- 71
They are in Russia, and a USA 4G proxy is about 120$. How many accounts can you run with that proxy?
blackgoldrider
Regular Member
- Jul 16, 2017
- 329
- 94
around 10-15 Lol
- Dec 31, 2017
- 3,516
- 2,686
so did any of you guys built the 4G proxies or still nothing ?
deeznutzca
Registered Member
- Sep 29, 2009
- 84
- 21
Anyone know of a place that builds these as a complete solution? Not interested in learning a new skill, but need the technical capability. Thx.
AWSProxies
Newbie
- Jun 29, 2019
- 32
- 11
You only need cell modems and unlimited USB ports on a bastion host. Run proxy servers on the bastion host and reverse tunnel to them from a public server. Use SSF (Secure Socket Funneling) instead of SSH for tunneling. SSF takes advantage of async I/O and threading.
Mian Nadeem
BANNED
- Jul 13, 2019
- 24
- 2
i think you need a ladder
fabioravila
Newbie
- Jul 25, 2019
- 5
- 5
Hello everybody
I'm new to the forum and since the end of 2017 I've been developing and looking for a 4g IP residential poll solution.
I come forward and apologize for my English and the size of the post.
I was surprised to see several techniques and ideas similar to the ones I applied or tried to apply.
Even the video of the Russian guy has the build similar to what I am going to do.
So I will share my experiences so far.
My Need: We run scrapper queries on government websites in my country (Brazil), services have limitations of + or - 30 queries per day per ip.
We do more than 9k searches per day.
So I need a poll of renewable IPs on demand where i can control the access programmatically on each one.
These websites block ip from other countries, cloud services with large amounts of IP of Brazil get very expensive and we can't afford it.
So a pool of renewable ips proxies is great, 4g allows a very fast ip refresh (from 6 to 20 seconds depending on the modem),
It is and much cheaper than several ADSL connections for example.
Because our don't use so much data, throttling is not a problem
I will start from my current attempt, which I am working on and that I am taking forward
Current Project - Multiple independents proxy nodes (split to conquer)
This is what I am currently working on is similar to the Russian video.
We will separate each proxy node into a set of independent hardware and software that can be connected to the common ethernet.
In my case:
1 x GL-AR150 router with openwrt
1 x HUAWEI E3272
Each openwrt, i do:
1-Configure the 4g modem (via gui, becouse i not linux expert)
2-Configure a 3proxy (or tinyproxy) with 3001 port
3-Adjust to connect
Esquema:
AR150_1 - 192.168.0.101 -> proxy: 3001 -> dongle_1
AR150_2 - 192.168.0.102 -> proxy: 3001 -> dongle_2
AR150_3 - 192.168.0.103 -> proxy: 3001 -> dongle_3
[...]
All connected on the same network with an internal router.
So within the network I have access to various proxies, each onde with a different internet connection.
To restart 4g (renew ip) on demand, as I understand it, there is even a command line that can be executed via SSH, I have not implemented it yet, but I believe it is not difficult to automate.
I am still building this solution, as soon as I have news I will post here.
It´s a work-in-progress, so i on the 3proxy step testing with 2 devices.
But, put proxy on AR150 can give some performance issue (is not a fast hardware)
So what I would like fot future is do different setup regarding connection management.
My idea is:
1) Use each AR150 as an internet gateway (no proxy)
2) Put all together on a switch, like i do now
3) Use a separate computer to manage all proxies and somehow each proxy use only a particular gateway.
So now
Well, for me this is the trick part.
Put a separate pc with multiple proxies is easy part,
but bind each proxy to specific gateway (AR150) on the network, i dont know how do this
For this, as i know i need for each proxy one NIC with specific gateway.
HOw can i do this (Windows or Linux)?
My future tries are:
On windows, tries to use some virtualization tecnique to add virtual NIC, with specification
On linux, some tellme that´s is possible add virutal NIC and configure some interface and route
Is it possible to have a single PC with multiple virtual NICs (one for each gateway) and create multiple bind proxies from each NIC?
I try to avoid put this on a VM.
On this aproach is easy add other WAN to the network, not necessarily 4G.
+PRO
-CON
+ Scaling is much easier since ethernet connections are made, just add HUBS
+ Isolating logic on a router helps a lot in setup and troubleshooting
- More hardware and more expensive initially
- I still need know how keep all the proxies on sigle PC, becouse AR150 performance is poor
Below my past attempts for anyone who wants to see
==================== Past Attempts ====================
1) Multiple 4g Dongle on USB Hub, in a single computer
As spoken here by a member, is to put a UBS HUB and add several 4g modems
Configure ppp connections and proxy instances (3proxy or tinyproxy), each with a binding port on a dongle.
I used this concept as POC, and made all the settings manually.
As I do not understand much of linux I could not set up an automated scripting system.
It worked great, with 5 dongles in my test, I ran the scripts manually, ppp and wvdial reconnected alone
Thus we have for example:
192.168.0.100:3001 -proxy-> dongle 1
192.168.0.100:3002 -proxy-> dongle 2
192.168.0.100:3003 -proxy-> dongle 3
[...]
To reset the modems I just had to kill the ppp process that wvdial would call again with a new IP.
Someone who understands linux and bash could manage this automatically with tracing and ssh commands or a simple webservice to reboot.
With this I proved that it was possible to carry out the venture, but the whole system seemed very fragile, a single problem with usb brought a huge headache.
And at some point it was necessary to restart the whole system. usb_switchmode didn't always work correctly, so scaling was more complicated, even in setup.
2) Multiple 4g USB HUB Dongles, single computer with VM for each proxy and USB
This second attempt is equals the first, but i put each proxy on a individual VM with a USB binding.
So each VM has only one USB and one NIC, and made the system more easy to configure
VM_1 -> UsbBind -> HOST -> RealUsb -> dongle_1
VM_2 -> UsbBind -> HOST -> RealUsb -> dongle_2
VM_3 -> UsbBind -> HOST -> RealUsb -> dongle_3
That it, i still work on this solution, so far i have news i´ll post here
I'm new to the forum and since the end of 2017 I've been developing and looking for a 4g IP residential poll solution.
I come forward and apologize for my English and the size of the post.
I was surprised to see several techniques and ideas similar to the ones I applied or tried to apply.
Even the video of the Russian guy has the build similar to what I am going to do.
So I will share my experiences so far.
My Need: We run scrapper queries on government websites in my country (Brazil), services have limitations of + or - 30 queries per day per ip.
We do more than 9k searches per day.
So I need a poll of renewable IPs on demand where i can control the access programmatically on each one.
These websites block ip from other countries, cloud services with large amounts of IP of Brazil get very expensive and we can't afford it.
So a pool of renewable ips proxies is great, 4g allows a very fast ip refresh (from 6 to 20 seconds depending on the modem),
It is and much cheaper than several ADSL connections for example.
Because our don't use so much data, throttling is not a problem
I will start from my current attempt, which I am working on and that I am taking forward
Current Project - Multiple independents proxy nodes (split to conquer)
This is what I am currently working on is similar to the Russian video.
We will separate each proxy node into a set of independent hardware and software that can be connected to the common ethernet.
In my case:
1 x GL-AR150 router with openwrt
1 x HUAWEI E3272
Each openwrt, i do:
1-Configure the 4g modem (via gui, becouse i not linux expert)
2-Configure a 3proxy (or tinyproxy) with 3001 port
3-Adjust to connect
Esquema:
AR150_1 - 192.168.0.101 -> proxy: 3001 -> dongle_1
AR150_2 - 192.168.0.102 -> proxy: 3001 -> dongle_2
AR150_3 - 192.168.0.103 -> proxy: 3001 -> dongle_3
[...]
All connected on the same network with an internal router.
So within the network I have access to various proxies, each onde with a different internet connection.
To restart 4g (renew ip) on demand, as I understand it, there is even a command line that can be executed via SSH, I have not implemented it yet, but I believe it is not difficult to automate.
I am still building this solution, as soon as I have news I will post here.
It´s a work-in-progress, so i on the 3proxy step testing with 2 devices.
But, put proxy on AR150 can give some performance issue (is not a fast hardware)
So what I would like fot future is do different setup regarding connection management.
My idea is:
1) Use each AR150 as an internet gateway (no proxy)
2) Put all together on a switch, like i do now
3) Use a separate computer to manage all proxies and somehow each proxy use only a particular gateway.
So now
Well, for me this is the trick part.
Put a separate pc with multiple proxies is easy part,
but bind each proxy to specific gateway (AR150) on the network, i dont know how do this
For this, as i know i need for each proxy one NIC with specific gateway.
HOw can i do this (Windows or Linux)?
My future tries are:
On windows, tries to use some virtualization tecnique to add virtual NIC, with specification
On linux, some tellme that´s is possible add virutal NIC and configure some interface and route
Is it possible to have a single PC with multiple virtual NICs (one for each gateway) and create multiple bind proxies from each NIC?
I try to avoid put this on a VM.
On this aproach is easy add other WAN to the network, not necessarily 4G.
+PRO
-CON
+ Scaling is much easier since ethernet connections are made, just add HUBS
+ Isolating logic on a router helps a lot in setup and troubleshooting
- More hardware and more expensive initially
- I still need know how keep all the proxies on sigle PC, becouse AR150 performance is poor
Below my past attempts for anyone who wants to see
==================== Past Attempts ====================
1) Multiple 4g Dongle on USB Hub, in a single computer
As spoken here by a member, is to put a UBS HUB and add several 4g modems
Configure ppp connections and proxy instances (3proxy or tinyproxy), each with a binding port on a dongle.
I used this concept as POC, and made all the settings manually.
As I do not understand much of linux I could not set up an automated scripting system.
It worked great, with 5 dongles in my test, I ran the scripts manually, ppp and wvdial reconnected alone
Thus we have for example:
192.168.0.100:3001 -proxy-> dongle 1
192.168.0.100:3002 -proxy-> dongle 2
192.168.0.100:3003 -proxy-> dongle 3
[...]
To reset the modems I just had to kill the ppp process that wvdial would call again with a new IP.
Someone who understands linux and bash could manage this automatically with tracing and ssh commands or a simple webservice to reboot.
With this I proved that it was possible to carry out the venture, but the whole system seemed very fragile, a single problem with usb brought a huge headache.
And at some point it was necessary to restart the whole system. usb_switchmode didn't always work correctly, so scaling was more complicated, even in setup.
2) Multiple 4g USB HUB Dongles, single computer with VM for each proxy and USB
This second attempt is equals the first, but i put each proxy on a individual VM with a USB binding.
So each VM has only one USB and one NIC, and made the system more easy to configure
VM_1 -> UsbBind -> HOST -> RealUsb -> dongle_1
VM_2 -> UsbBind -> HOST -> RealUsb -> dongle_2
VM_3 -> UsbBind -> HOST -> RealUsb -> dongle_3
That it, i still work on this solution, so far i have news i´ll post here
tadeuonbrand
Regular Member
- Jun 30, 2012
- 307
- 97
15 IG proxies. You have to believe you have a private 4G Mobile !
thebusboss
Newbie
- Aug 3, 2019
- 2
- 0
Why is hilink or any web interface must have? is it just for restarting device (rotating proxy/changing ip?), if that's what it's all about, it could be done running a script unplugging device or even a physical disconnection and connection of the device, am I right?
SocialMediaNetwork
Regular Member
- Jan 24, 2019
- 490
- 253
This because there are also dongles that you have to dial to get them online and those with web interface you don't have to dial them to get them online.
those dongles that you have to dial are headache! avoid them and always get one that has web interface.and yes you can always run a script to unplug device and reconnect to change ip.
theeyeinside
Senior Member
- Aug 19, 2014
- 962
- 867
All this is described in the book called "Proxy Know 3.0" in the BHW misc marketplace section. I hear it includes the proxy script for Raspberry Pi and walks you through setting everything up.
Sincerely,
TheEye
Sincerely,
TheEye
I did system use 1 Pi Zero W run with 7 dongles,and has API for get proxy and change IP.
I see that is ok.Does anyone know where can buy cheap dongle 3g/4g ? I tested with my dongles use 3G, ping so high.
View attachment 119007
Last edited by a moderator:
thebusboss
Newbie
- Aug 3, 2019
- 2
- 0
check whats your current signal strength
you can do that with comgt -d /dev/ttyUSBX i believe
also how far you are from base transceiver stations
tadeuonbrand
Regular Member
- Jun 30, 2012
- 307
- 97
I did system use 1 Pi Zero W run with 7 dongles,and has API for get proxy and change IP.
Does anyone know where can buy cheap dongle 3g/4g ? I tested with my dongles use 3G, ping so high.
View attachment 119007
Thats trully amazing. Are you connecting it where ? Directly into your computer ?
Can Facebook/Instagram detect that you are using 4G modem to use it on their networks ?
On the internet, can connect via my Wan IP (Public IP). cause I Nat port for my device
But I surprise when in my LAN, I can connect to my Public IP too. I think that is cannot, so I don't understand how I can connect to my public IP
I use 3g dongles for cheaps.
Cause it's enough for my task
pr0ject1
Regular Member
- Feb 12, 2018
- 454
- 61
A dedicated US 4G proxy should not cost that much. Like half that is the max they would go for.
fabioravila
Newbie
- Jul 25, 2019
- 5
- 5
I did system use 1 Pi Zero W run with 7 dongles,and has API for get proxy and change IP.
Does anyone know where can buy cheap dongle 3g/4g ? I tested with my dongles use 3G, ping so high.
View attachment 119007
That´s great, Could you share your code and scripts?
Sorry i can't share them. My system has 4 parts
- Backend
- Api (get proxy / change ip)
- Software manager
- Scripts for Raspbian
Sorry i can't share them. My system has 4 parts
- Backend
- Api (get proxy / change ip)
- Software manager
- Scripts for Raspbian
I have just upgrade my system , 5 pi zero run with 35 dongles