What is high concurrency ?

CamelCase

Junior Member
Joined
Oct 21, 2019
Messages
192
Reaction score
32
The company where I bought a proxy offered me such an option, but I did not understand exactly what it did because of the technical statements?

What will benefit me if I buy high concurrency? Will it increase the speed of my proxies?
 
The speed of a proxy is given. High concurrency is related to the limit of concurrent connections you can have per ip address to a proxy. Concurrent connections mean multiple simultaneous threads. Let's say a provider allows 100 concurrent connections or threads per proxy, it means that you can run a capable multithreaded tool with up to 100 threads per proxy at the same time, e.g. you can run a scraper with up to 100 threads per proxy simultaneously.
 
The speed of a proxy is given. High concurrency is related to the limit of concurrent connections you can have per ip address to a proxy. Concurrent connections mean multiple simultaneous threads. Let's say a provider allows 100 concurrent connections or threads per proxy, it means that you can run a capable multithreaded tool with up to 100 threads per proxy at the same time, e.g. you can run a scraper with up to 100 threads per proxy simultaneously.

Hey man thanks for answer. My technical knowledge is not enough for me to understand what you say. I have a scenario like this:
I have 100 proxies and I pull data from the site to the target. I use IP addresses in order in each request. I have a two question.
1- There is an API limit on the site where I draw the data. Will it affect the limit when I buy this feature?
2- Will My Proxies pull data faster?
 
Hey man thanks for answer. My technical knowledge is not enough for me to understand what you say. I have a scenario like this:
I have 100 proxies and I pull data from the site to the target. I use IP addresses in order in each request. I have a two question.
1- There is an API limit on the site where I draw the data. Will it affect the limit when I buy this feature?
2- Will My Proxies pull data faster?
So you have 100 proxies with let's say a limit of 100 simultaneous threads each. That's 100x100 threads, which would obviously make the scraping faster than only 100 threads (1 per proxy), but sites usually apply rate limit, when you scrape with many threads per ip address. This limit is usually something you won't know, until you hit it or it's publicly available somewhere.
 
Hey man thanks for answer. My technical knowledge is not enough for me to understand what you say. I have a scenario like this:
I have 100 proxies and I pull data from the site to the target. I use IP addresses in order in each request. I have a two question.
1- There is an API limit on the site where I draw the data. Will it affect the limit when I buy this feature?
2- Will My Proxies pull data faster?

Do you use it for web scraping?
1. No. API limited, So based website API to set your threads for each proxy.
2. Yes. More proxies for "concurrency", so you can get more faster.
 
I have 100 proxies and I pull data from the site to the target. I use IP addresses in order in each request. I have a two question.
1- There is an API limit on the site where I draw the data. Will it affect the limit when I buy this feature?
2- Will My Proxies pull data faster?

1. It depends:
a. YES: If the API is open (you don't need a key to login) and the website can't identify your exact requests, then if you connect through different IP address the API website won't be able to match API requests with you.
b NO: If the API si closed and you need an account and an auth key to access it, then it doesn't matter how many proxies you'll be using because the request counting and limit is enforced on your API account. However, you can bypass this limit by creating multiple API accounts (auth keys) and then rotating these keys through your proxies.

2. Generally yes, you can scale your requests number with proxies, but you need a multi-threaded app to do this.
 
Back
Top