Looking for free/cheap proxy scraper and tester

NorbGier

Junior Member

May 30, 2020

124

61

• Jul 1, 2020

• #1

I realized that I use only 30% of my virtual machine's CPU while scraping. I want to utilize unused so running more threads will be the best option.
I can either buy more proxies which is not worth it for me or use public scraped proxies.
So I decided to run two parallel scrapers one with private proxies and the other one with public proxies. I need public proxy scraper and tester that can run continuously in the background and feed my scraper with fresh proxies.
As in thread name I am looking for free or cheap proxy scraper and tester.

 

William Parsons

BANNED

Mar 16, 2020

60

24

• Jul 1, 2020

• #2

Is using a VM with lower specs instead of using resources to scrape low-quality proxy lists an option?

If not, there are some free public proxy API services available if you search "free proxy API" (they already do both the scraping and proxy testing).

 

NorbGier

Junior Member

May 30, 2020

124

61

• Jul 1, 2020
• Thread Starter Thread Starter

• #3

I don't pay for these vms as I use free trials on azure/vultr/aws. Free credits lats 30 days so lowering specs won't save me any money.

 

Whitewhistle

Power Member

Aug 2, 2018

547

113

• Jul 2, 2020

• #4

What are your target(s)? It isnt often the case that free proxies work while scraping something with more protection. But it's great, if it works for you!

 

turelink

Elite Member

Jr. VIP

Jul 26, 2015

2,625

711

• Jul 2, 2020

• #5

Try GSA proxy scraper!

 

proxygo

Elite Member

Executive VIP

Jr. VIP

Nov 2, 2008

59,130

26,640

• Jul 2, 2020

• #6

for a free proxy tester use Bleach proxy checker
with your own proxy judge , but it only checks for
anon / transparent proxies

 

NorbGier

Junior Member

May 30, 2020

124

61

• Jul 5, 2020
• Thread Starter Thread Starter

• #7

I have tested some scrapers and ended buying 250 shared proxies for 6$ and surprisingly they are working just slightly worse than my private google cloud proxies. Also I set up chat bot on that vps to fill cpu usage even more.

 

KocerRoxy

Power Member

Jr. VIP

Sep 15, 2010

512

179

• Sep 27, 2020

• #8

Unless your software is configured to easily rotate through many proxies. Using scraped proxies is not worth it any more. The time your software spends blocked with non-functional proxies is just wasted cpu cycles.

 

proxygo

Elite Member

Executive VIP

Jr. VIP

Nov 2, 2008

59,130

26,640

• Sep 27, 2020

• #9

@m4dm4n most of these 2/3 month threads
you keep bumping has been answered

 

KocerRoxy

Power Member

Jr. VIP

Sep 15, 2010

512

179

• Nov 6, 2020

• #10

I'm actually looking at most of them for ideas for content. And I only add answers to them if I have a perspective that I think is at the same time valuable and also not already present on the thread.

Thank you for your input.