Anyone scraping Amazon here on a large scale?

[jamie3000]

Anyone scraping Amazon here on a large scale?

I have some questions about 503 errors and JS captcha execution specific to Amazon.

Would be great to pick your brain with some tricky questions

 

[thetrustedzone]

i coded few amazon scrapers before , that error probably due to missed http header , use fiddler to capture and simulate all headers , if failed pm me your code i will check it for you .

 

[jamie3000]

i coded few amazon scrapers before , that error probably due to missed http header , use fiddler to capture and simulate all headers , if failed pm me your code i will check it for you .

The issue I'm having is about 70% of my proxies are being 503 blocked very quickly. Where as the other 30% work perfectly, they'll get the captcha challenge eventually then I'll solve it (my system) and they'll carry on fine.

 

[thetrustedzone]

The issue I'm having is about 70% of my proxies are being 503 blocked very quickly. Where as the other 30% work perfectly, they'll get the captcha challenge eventually then I'll solve it (my system) and they'll carry on fine.

seems you found root of problem and already solved it !

 

[Houdinii]

It's all about the headers and it's only on certain pages. It doesn't even seem to burn my ip addresses when I forget either. I think I'm gonna check out Amazon's product API tonight. I'm feeling inspired

 

[bartosimpsonio]

That was my money maker in 1999. With API's and shit you don't really need to scrape Amazon any more.

 

[Automation Guru]

I use my own custom scraper I built for amazon.. I have no issues with 503 errors and JS captcha execution as of yet..

 

[yourdady]

I use my own custom scraper I built for amazon.. I have no issues with 503 errors and JS captcha execution as of yet..

Would you mind sharing?

 

[Automation Guru]

Would you mind sharing?

You want me to share my custom bot? NO - That is why it is custom.. i.e for me only

 

[yourdady]

You want me to share my custom bot? NO - That is why it is custom.. i.e for me only

Share is caring...!

 

[David AffForce]

i also have this problem