1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Is it possible to scrape all products and pricing of an ecommerce site?

Discussion in 'Black Hat SEO' started by SuperNoobInc, Jan 6, 2012.

  1. SuperNoobInc

    SuperNoobInc Regular Member

    Joined:
    Dec 28, 2011
    Messages:
    467
    Likes Received:
    32
    Is it possible to scrape all products and pricing of an ecommerce site, and organize them on an excel file - so I can study competitor's pricing?
     
  2. curlytorn

    curlytorn Newbie

    Joined:
    Jul 8, 2011
    Messages:
    18
    Likes Received:
    0
    I believe so...
     
  3. SuperNoobInc

    SuperNoobInc Regular Member

    Joined:
    Dec 28, 2011
    Messages:
    467
    Likes Received:
    32
    Thanks.
    How can we determine if its populated via ajax or not?
     
  4. m3ownz

    m3ownz Regular Member

    Joined:
    Dec 12, 2009
    Messages:
    311
    Likes Received:
    135
    Yup, difficulty and thus cost, depend on a number of things, primarily if all prices are available in the source code, its quite simple. If the prices are populated via ajax, its a lot harder
     
    • Thanks Thanks x 1
  5. SuperNoobInc

    SuperNoobInc Regular Member

    Joined:
    Dec 28, 2011
    Messages:
    467
    Likes Received:
    32
    Any suggestions? Can you please elaborate?
     
  6. m3ownz

    m3ownz Regular Member

    Joined:
    Dec 12, 2009
    Messages:
    311
    Likes Received:
    135
    Well load up a product page and right click > view source.

    If you see the item / price combos in the source, its a simple matter of scraping the source code and parsing out the relevent info.

    If your unsure, feel free to PM me a link
     
  7. ILoveCSharp

    ILoveCSharp Regular Member

    Joined:
    Nov 2, 2011
    Messages:
    368
    Likes Received:
    45
    Right now i am doing that for my master degree. I am writing individual scraper code for each website. It is really hard and you have to be experienced with coding otherwise forget it :) (location Turkey)
     
  8. Michaelf

    Michaelf Registered Member

    Joined:
    Nov 17, 2011
    Messages:
    59
    Likes Received:
    11
    Of course it's possible. You can use make bots for it with programs like, zennoposter,ubot or program it in a language of your choice.
     
  9. purewealthinc

    purewealthinc Regular Member

    Joined:
    May 3, 2010
    Messages:
    427
    Likes Received:
    383
    Occupation:
    Web Fishing
    Location:
    World Wide Web City
    It's possible mate specially if you want only to extract the Product name and product prizes and save into excel..

    But anyway, are trying trying to scrape/extract the product name and prizes only and save into excel with Colum A, B, C and so on...? If so.. that's pretty simple as 1-2-3.
     
  10. pirondi

    pirondi Power Member

    Joined:
    Jan 5, 2010
    Messages:
    562
    Likes Received:
    118
    If the info is on the source code you just need to use scrape softwares to do it.

    Just look for 'web extractor' on google you will find some,the one i use is from webextract.net

    If the information is not on source code,you would need to hire someone to make a bot to copy/paste the info.
     
    Last edited: Jan 6, 2012
  11. SuperNoobInc

    SuperNoobInc Regular Member

    Joined:
    Dec 28, 2011
    Messages:
    467
    Likes Received:
    32
    I went to their page and viewed their source; I am able to see their product name and pricing thre; something like this;

    If the above case is true, does that mean the site is NOT in ajax and data can be easily scraped?
     
  12. purewealthinc

    purewealthinc Regular Member

    Joined:
    May 3, 2010
    Messages:
    427
    Likes Received:
    383
    Occupation:
    Web Fishing
    Location:
    World Wide Web City
    Even if the the product name is not in the source code, we can still scrape the Product Name and Prizes in the website.. We don't need the source code, it doesn't matter if there's no data in the source code, we can scrape that in the website.

    If you are looking someone to do that, just let me see the website, pm me if you are still looking for someone to do that!