1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Myth or Real? Amazon entire product catalog...

Discussion in 'BlackHat Lounge' started by amsteve, Jan 8, 2009.

  1. amsteve

    amsteve Newbie

    Joined:
    Nov 25, 2008
    Messages:
    26
    Likes Received:
    6
    I was speaking with a friend the other day and he mentioned he was hunting for an XML file that contained Amazon's entire catalog of products. He said he had seen it discussed before on certain forums and that all he knows is it's a XML file and about 1GB in size.

    I was curious if anyone had access to this or knew how to get a hold of it, assuming it exists.

    If for some reason we can't find the full product catalog we are looking for something that (scraper) will systematically go page by page and gather all of the information from each product including short description, long description, prices, images, urls, etc. for the jewelry section.

    If anyone can be of assistance it would be much appreciated, depending on what you have we might be willing to pay.

    Thanks.
     
  2. HaRRo

    HaRRo Elite Member

    Joined:
    Oct 29, 2005
    Messages:
    2,676
    Likes Received:
    13,447
    Occupation:
    Self Employed
    Location:
    Miami, FL
    • Thanks Thanks x 1
  3. Lombi

    Lombi Registered Member

    Joined:
    Jan 13, 2008
    Messages:
    53
    Likes Received:
    10
    It's far more than one gig. Allposters has a datafeed that's one gig and it's 350.000 posters. Even if you just do books it's 24 MILLION items.

    Amazon can give you a datafeed if you ask them.

    But what they also give you is an access to their XML API which contains everything. Amazon associates, google it :)
     
  4. amsteve

    amsteve Newbie

    Joined:
    Nov 25, 2008
    Messages:
    26
    Likes Received:
    6
    Ok, he does belong to AWS already, but he is having a problem getting it to do what he needs and he is no newbie. Possibly we'll contact Amazon support.