Using Amazon EC2 as a proxy?

soapradio

Newbie
Joined
Aug 9, 2011
Messages
2
Reaction score
0
I'm considering using Amazon EC2 as a proxy to scrape a website's live data through PHP/cURL. The website throttles based on IP. There are thousands of requests involved.. The requests need to be made live (as users search on my site) because the data changes throughout the day.

Is Amazon EC2 a good choice for a proxy here? Will scraping on their service draw Amazon's ire?

Note: This site isn't scraped by many people, so they're definitely not setup to block Amazon IP's specifically.
 
I use ec2 regularly as a proxy for scraping google (keyword research). It actually works really well. I haven't had any complaints from amazon.

At times it can take up to a minute to spin up an instance, so you would want to have a background job to do that. The tiny instances work really well as squid proxies and are cheap.

Also, if you go with squid be sure to use the latest version, since older versions aren't completely transparent.
 
Thanks napoleon007.Will check this out
 
Last edited:
Back
Top