Python script bandwidth

manas23

manas23

Junior Member
Joined
May 24, 2018
Messages
173
Reaction score
31
I have been running 10 python scripts on my personal computer. I have tried with dedicated server as well(1000mbps).
With my personal pc, running 10 scripts at the same time, I am able to process 250 requests a hour each script.
On dedicated, I was able do to 500 each script. My question is; how can I double the scripts without losing performance? The script downloads a picture and uploads.
I can double the scripts on my personal computer but they do 120 process each.
 
Advertise on BHW
What does the throttle comes from?

Is it from your PC's resources, from your Internet connection or from your IP address being blocked?

If you can provide some info on the actual issue throttling your scripts, you might get better help.
 
HydraProxy said:
What does the throttle comes from?

Is it from your PC's resources, from your Internet connection or from your IP address being blocked?

If you can provide some info on the actual issue throttling your scripts, you might get better help.
Click to expand...
I think it is from the speed of my internet. But like I said when you compare to 50mbps to 1000mbps, It just doubles the processes they make. Do you think creating another windows 10's and running the scripts partly would solve my problem?
 
Without knowing what you're doing exactly, it's hard to give a proper answer.

In all likelihood, however, the limit is probably not the speed of the connection, but the network latency, the speed of the server responding to your requests, and the time spent negotiating connections (which is also affected by the latency). Moving to a data centre probably knocked down the network latency component.

Now, assuming you're not saturating the bandwidth of the connection, what you probably want is to look at implementing concurrency, either asynchronously (in python with the treq + twisted libraries, for example), or with threading. The first option is generally preferable, but has a bit of a steeper learning curve.

What this will do is allow each script to make multiple requests all at once, instead of making them one at a time in sequence.
 
I mean if you can't modify the source code of the script (since its an exe) your options are limited.

You can
1) Either launch the exe process multiple times
2) Ask the guy to add multithreading to the script
 
manas23 said:
Do you think creating another windows 10's and running the scripts partly would solve my problem?
Click to expand...

Yes, if the problem is your Internet speed, why not mitigate it by getting a Windows VPS residing on a fast-connection datacenter.

You can check the marketplace for windows VPS, there are some good offers there at good prices.
 
The answers are suggesting adding multithreading or multiprocessing, but if you are already running multiple instances of the script at the same time then that isn't an issue.

It sounds like when you are running it on your personal connection, you are limited by your 50mbps Internet speed, so when you double the number of scripts that are running, each one only has half as much Internet bandwidth available to it.

What happens on the server if you double the number of scripts running? If it doesn't roughly double the total requests then I suspect either a) the server isn't actually providing 1000mbps or b) the server is running out of CPU capacity. In this case try getting a server with more cores and see if that helps.
 
Gotta find the bottleneck. Is it the network speed? Or is it the PC or something else? If network speed then get a faster internet. Otherwise if PC then you'll need to run the script in multiple machines or something like that.
 
Ask to the guy for the source code and someone could optimize it for u most of the time it could be just a matter of poor source code optimization
 
Are you multithreading? you should take a look at that. You can make multiple threads and make script fast af
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top