Amazon AutoBlog - Is there any tools / plugin which can pull huge amount of amazon product
- Thread starter trungpt2
- Start date
suradreamz
Power Member
- Jun 30, 2013
- 517
- 50
You cannot upload that kind of (200K) Products without VPS Hosting plan...!
wpbacklinks
Elite Member
- Mar 27, 2010
- 3,762
- 1,499
are you using shared hosting?
You're going to run into a lot of problems with Wordpress when trying to create that many pages/posts.
The Wordpress admin area sucks for managing large quantities of pages/posts - making bulk changes to that many pages is near impossible in Wordpress, the interface isn't designed for that. Even when using some of the plugins for bulk editing, they will error out on you when you're working with thousands of pages/posts. I'm trying to upload about 20k posts currently, on a shared hosting plan, and I can only do about 1000 at a time - otherwise it just clams up on me and only partially imports my spreadsheet, if at all.
There is a program, which has been shared on here, called WebHarvy, which can scrape Amazon - but it's really slow, and if a page takes too long to load during the process, it causes errors and you basically need to start all over.
Also, even if you could scrape all these products into Wordpress - how do you plan to manage them as they come and go? Products are constantly being added and removed from Amazon, running out of stock, changing in price, etc...
Amazon has a categorization of it's products known as "nodes". There are some plugins (and standalone scripts) out there which use these Amazon nodes to pull products into your web site on the fly, which may be a better option for you, especially since they update dynamically as well. Even then, though, I believe they can only pull in like the first 10 pages worth of products - due to some limitation with the Amazon APIs
The Wordpress admin area sucks for managing large quantities of pages/posts - making bulk changes to that many pages is near impossible in Wordpress, the interface isn't designed for that. Even when using some of the plugins for bulk editing, they will error out on you when you're working with thousands of pages/posts. I'm trying to upload about 20k posts currently, on a shared hosting plan, and I can only do about 1000 at a time - otherwise it just clams up on me and only partially imports my spreadsheet, if at all.
There is a program, which has been shared on here, called WebHarvy, which can scrape Amazon - but it's really slow, and if a page takes too long to load during the process, it causes errors and you basically need to start all over.
Also, even if you could scrape all these products into Wordpress - how do you plan to manage them as they come and go? Products are constantly being added and removed from Amazon, running out of stock, changing in price, etc...
Amazon has a categorization of it's products known as "nodes". There are some plugins (and standalone scripts) out there which use these Amazon nodes to pull products into your web site on the fly, which may be a better option for you, especially since they update dynamically as well. Even then, though, I believe they can only pull in like the first 10 pages worth of products - due to some limitation with the Amazon APIs
Hi,
I also have something similar like amazon autoblog. I have custom website running on django. I run my update code every hour on my server. Currently my db has 200 000 items. I did not made any links to my site, but I already have 3 ordered items with 4 percent coversion rate. But it's really hard to handle that large db. My search files have 200MB and I currently am struggling to make it work. I plan on making smaller database with more expensive products. btw, my db also takes ages to load
I also have something similar like amazon autoblog. I have custom website running on django. I run my update code every hour on my server. Currently my db has 200 000 items. I did not made any links to my site, but I already have 3 ordered items with 4 percent coversion rate. But it's really hard to handle that large db. My search files have 200MB and I currently am struggling to make it work. I plan on making smaller database with more expensive products. btw, my db also takes ages to load