Python script bandwidth

manas23

Junior Member
Joined
May 24, 2018
Messages
165
Reaction score
28
I have been running 10 python scripts on my personal computer. I have tried with dedicated server as well(1000mbps).
With my personal pc, running 10 scripts at the same time, I am able to process 250 requests a hour each script.
On dedicated, I was able do to 500 each script. My question is; how can I double the scripts without losing performance? The script downloads a picture and uploads.
I can double the scripts on my personal computer but they do 120 process each.
 

HydraProxy

Jr. VIP
Jr. VIP
Joined
Jun 6, 2020
Messages
510
Reaction score
232
Website
hydraproxy.com
What does the throttle comes from?

Is it from your PC's resources, from your Internet connection or from your IP address being blocked?

If you can provide some info on the actual issue throttling your scripts, you might get better help.
 

manas23

Junior Member
Joined
May 24, 2018
Messages
165
Reaction score
28
What does the throttle comes from?

Is it from your PC's resources, from your Internet connection or from your IP address being blocked?

If you can provide some info on the actual issue throttling your scripts, you might get better help.
I think it is from the speed of my internet. But like I said when you compare to 50mbps to 1000mbps, It just doubles the processes they make. Do you think creating another windows 10's and running the scripts partly would solve my problem?
 

wow

Registered Member
Joined
Dec 6, 2016
Messages
52
Reaction score
47
Without knowing what you're doing exactly, it's hard to give a proper answer.

In all likelihood, however, the limit is probably not the speed of the connection, but the network latency, the speed of the server responding to your requests, and the time spent negotiating connections (which is also affected by the latency). Moving to a data centre probably knocked down the network latency component.

Now, assuming you're not saturating the bandwidth of the connection, what you probably want is to look at implementing concurrency, either asynchronously (in python with the treq + twisted libraries, for example), or with threading. The first option is generally preferable, but has a bit of a steeper learning curve.

What this will do is allow each script to make multiple requests all at once, instead of making them one at a time in sequence.
 

Scr3aM-_-

Registered Member
Joined
Feb 18, 2017
Messages
52
Reaction score
20
I mean if you can't modify the source code of the script (since its an exe) your options are limited.

You can
1) Either launch the exe process multiple times
2) Ask the guy to add multithreading to the script
 

Klopferr

Jr. VIP
Jr. VIP
Joined
Sep 8, 2016
Messages
109
Reaction score
22
Use threading or start the .exe more often i guess
 

HydraProxy

Jr. VIP
Jr. VIP
Joined
Jun 6, 2020
Messages
510
Reaction score
232
Website
hydraproxy.com
Do you think creating another windows 10's and running the scripts partly would solve my problem?

Yes, if the problem is your Internet speed, why not mitigate it by getting a Windows VPS residing on a fast-connection datacenter.

You can check the marketplace for windows VPS, there are some good offers there at good prices.
 

rb32

Registered Member
Joined
Oct 14, 2020
Messages
53
Reaction score
20
The answers are suggesting adding multithreading or multiprocessing, but if you are already running multiple instances of the script at the same time then that isn't an issue.

It sounds like when you are running it on your personal connection, you are limited by your 50mbps Internet speed, so when you double the number of scripts that are running, each one only has half as much Internet bandwidth available to it.

What happens on the server if you double the number of scripts running? If it doesn't roughly double the total requests then I suspect either a) the server isn't actually providing 1000mbps or b) the server is running out of CPU capacity. In this case try getting a server with more cores and see if that helps.
 

bartosimpsonio

Jr. Executive VIP
Jr. VIP
Joined
Mar 21, 2013
Messages
20,286
Reaction score
23,946
Website
www.theglobeandmail.com
Gotta find the bottleneck. Is it the network speed? Or is it the PC or something else? If network speed then get a faster internet. Otherwise if PC then you'll need to run the script in multiple machines or something like that.
 

skule

Power Member
Joined
Mar 25, 2020
Messages
733
Reaction score
682
Ask to the guy for the source code and someone could optimize it for u most of the time it could be just a matter of poor source code optimization
 

AdvertiserLeads

Jr. VIP
Jr. VIP
Joined
Apr 15, 2020
Messages
163
Reaction score
19
Are you multithreading? you should take a look at that. You can make multiple threads and make script fast af
 
Top