I have been working on a mid-scale scraping project (~=2M-3M requests/day), and have been able to get by on a small pool of fixed private proxies as well as a proxy rotation service where I route most of my requests. As time wore on, different sites have implemented different anti-scraping...
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.