Sebastian Rooks
Junior Member
- Nov 16, 2015
- 138
- 113
Hi Black Hat World, it's been a long time.
tldr; I need data from the ebay feed api that requires membership in the ebay Partner Network.
For the past couple of months, I've been working on a dropshipping project, from a big box retailer to ebay.
I understand that there are many tools available for that sort of thing, but I'm a roll my own kind of guy.
Here's a brief outline of my approach:
1) I've built a database of around 4 million products from a retailer
2) Through a series of api keys, I can fully refresh that table in around 24 hours
3) I manage my ebay account through their api. So far just listing, I haven't automated fulfillment yet
Here's where I could use a hand....
4) I also have a table of ebay listings, which reflect the products the retailer has to offer, so that I can
use that data to determine what I want to list.
Ebay's search api rate limits don't allow for anywhere near the level of data I'd like to amass, - and then maintain.
So, I use a multithreaded system of proxies to scrape the data. On a good day, I can process up to around 10k items in an hour.. I also used a terabyte of bandwidth last month.
It works, but it's not the most efficient or elegant solution, and I aim to scale.
Ebay has the answer I need. It's all right there in the feed api. If I could get regular access to that data, (and I mean all, or most of it, daily) it could really help get me off the ground.
But that level of access is only granted to the Partner Network, of which I'm not a member.
If you've read this far down, then maybe you are?
I'd think that those with legitimate access to that data feed, don't want it for what I want it for. So, we're
not exactly in competition with one another. I'm not sure what sort of response I'm expecting, or even looking for, but I'm open to any thoughts or suggestions.
Thanks for reading,
-sebastian
mods, I hope this is in the right place. Sorry if it isn't.
tldr; I need data from the ebay feed api that requires membership in the ebay Partner Network.
For the past couple of months, I've been working on a dropshipping project, from a big box retailer to ebay.
I understand that there are many tools available for that sort of thing, but I'm a roll my own kind of guy.
Here's a brief outline of my approach:
1) I've built a database of around 4 million products from a retailer
2) Through a series of api keys, I can fully refresh that table in around 24 hours
3) I manage my ebay account through their api. So far just listing, I haven't automated fulfillment yet
Here's where I could use a hand....
4) I also have a table of ebay listings, which reflect the products the retailer has to offer, so that I can
use that data to determine what I want to list.
Ebay's search api rate limits don't allow for anywhere near the level of data I'd like to amass, - and then maintain.
So, I use a multithreaded system of proxies to scrape the data. On a good day, I can process up to around 10k items in an hour.. I also used a terabyte of bandwidth last month.
It works, but it's not the most efficient or elegant solution, and I aim to scale.
Ebay has the answer I need. It's all right there in the feed api. If I could get regular access to that data, (and I mean all, or most of it, daily) it could really help get me off the ground.
But that level of access is only granted to the Partner Network, of which I'm not a member.
If you've read this far down, then maybe you are?
I'd think that those with legitimate access to that data feed, don't want it for what I want it for. So, we're
not exactly in competition with one another. I'm not sure what sort of response I'm expecting, or even looking for, but I'm open to any thoughts or suggestions.
Thanks for reading,
-sebastian
mods, I hope this is in the right place. Sorry if it isn't.