Amazon Scraper ???
- Thread starter cro-mag
- Start date
infierno76
Newbie
- Oct 11, 2012
- 9
- 0
There is a site called WPAmanuke, that has a javascript you can run in your browser (you just set it up as a bookmark). Basically, you go to Amazon, search for a given type of product, then run the script and it will pull all the ASIN numbers off the items listed on that page. The only catch is you have to do this one page at a time.
Just out of curiosity, why not just pull the products in through browse nodes, that's much simpler than gathering individual ASIN numbers?
Just out of curiosity, why not just pull the products in through browse nodes, that's much simpler than gathering individual ASIN numbers?
TZ2011
Senior Member
- Jun 26, 2011
- 1,095
- 1,130
You can scrape all links.
Attack sitemaps. I will give you few links just like example how that looks and to understand how much data is there.
Code:
http://www.amazon.com/sitemap-manual-index.xml
http://www.amazon.com/sitemap_dp_index.xml
http://www.amazon.com/sitemap_vendor_videos_us.xml
http://www.amazon.com/sitemap_vod_index.xml
http://www.amazon.com/sitemap_entity_index.xml.gz
http://www.amazon.com/sitemaps.US_detail_page_sitemap_mobile_index.xml.gz
http://www.amazon.com/sitemaps.US_detail_page_sitemap_desktop_index.xml.gzEach of those sitemaps is even more compressed and you have to open them. You can use firefox plugin " DownThemAll " to download them faster. Later you have to clean the codes from xml and get clean links to each product. At the end you will have about 65 Gb of text files with links. I guess that you didn't wanted to do it on so big scale ?
ScrapeboxWorker
Regular Member
- Jul 23, 2012
- 498
- 285
If you are using wordpress try WP ROBOT its nulled somewhere
TZ2011
Senior Member
- Jun 26, 2011
- 1,095
- 1,130
WP plugin Azon Builder is doing that, you mark what products to scrape, he is scraping pictures, reviews, prices, automatically making links with your aff ID and formatting everything like a post.
If you want to make it harder (just for fun and enjoy of learning) you can do something with cUrl ,it is doable to make list with links from where you want reviews, include it in rotating and making it to save scraped html in text file or folder on server.
If you want to make it harder (just for fun and enjoy of learning) you can do something with cUrl ,it is doable to make list with links from where you want reviews, include it in rotating and making it to save scraped html in text file or folder on server.
usong725
Regular Member
- Nov 7, 2011
- 335
- 74
some information can be pull directly from Amazon via API. but they have stopped providing the customer reviews. But i found a site that displays it same as in Amazon site. even the "stars", username, location, etc were displayed.
will gonna check the WP plugin Azon Builder and see if i can get something from there.
Thanks
will gonna check the WP plugin Azon Builder and see if i can get something from there.
Thanks
Mission Ashwamedh
Junior Member
- Jul 4, 2012
- 187
- 118
Number of products scraped depends upon how many proxies you have..
I have created a tool for that. I can make customization in that tool according to your needs..
heartnet02
BANNED
- Aug 27, 2012
- 547
- 98
I got a script from one of the threads here that lets me pull a lot of products from any niche. It only works on magento and can't remember the name of the script but it really works.
HealeyV3, I saw one of your earlier posts about being able to scrape Amazon to get product data. This can be real useful data for marketers. I'd be willing to pay you for the 1.5 mil product records you initially pulled down and possible pay for your code or pay for you to produce new data periodically. Private message me to discuss.
heartnet02
BANNED
- Aug 27, 2012
- 547
- 98
I have my amazon scrapper which I can share if the price is right. Pm me for all those who are interested.
I'm looking to buy an Amazon scraper. I'm new to the forum (and SEO), but appears I'm in the right place. Any help will be appreciated.