1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Suggestions for scraping content behind login

Discussion in 'BlackHat Lounge' started by HighRiskJohn, Jan 22, 2013.

  1. HighRiskJohn

    HighRiskJohn Junior Member

    Joined:
    Jan 18, 2012
    Messages:
    173
    Likes Received:
    101
    Occupation:
    Self-Employed
    Location:
    Chicago, IL
    I'm building out a couple of e-commerce sites and I want to quickly scrape the images and content descriptions to be able to import into an ecommerce site. I am an approved dealer, but one of hte sites won't give their product feed for free, they want to sell it. (It's expensive) and I only want 100-200 out of the 3,000+ they sell. I've tried using sitesucker, but it doesn't seem able to save the images. (Which is what I really want)

    Another site i'm building to promote my wife's business also doesn't pull in any of the product images.

    Any suggestions on how to scrape the site at least for the images? The content I can deal with, but it's the images that I really want.
     
  2. HighRiskJohn

    HighRiskJohn Junior Member

    Joined:
    Jan 18, 2012
    Messages:
    173
    Likes Received:
    101
    Occupation:
    Self-Employed
    Location:
    Chicago, IL
    I forgot to ask, would this be something that uBot could do? Meaning, save the image url to a column in a csv file and same the description into another column then save the retail price into another column and the wholesale price into another column?
     
  3. HighRiskJohn

    HighRiskJohn Junior Member

    Joined:
    Jan 18, 2012
    Messages:
    173
    Likes Received:
    101
    Occupation:
    Self-Employed
    Location:
    Chicago, IL
    No one knows?