1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How do I scrape databases?

Discussion in 'Black Hat SEO Tools' started by Electricfan, Dec 9, 2009.

  1. Electricfan

    Electricfan Newbie

    Joined:
    Feb 6, 2009
    Messages:
    3
    Likes Received:
    3
    Hi all,
    I am trying to scrape some databases to save some time... The pages are coming up with as http://shop.mywebgrocer.com/Browse.aspx?&sid=34474644&sid_guid=f28e0365-7e5e-4e04-8c10-6bd0101b903e&strid=DD39796&catL0=570&catL1=585&catL2=1377&catL3=1015&HasProducts=1


    Does this help? It's a site that has grocery store items on it..

    Electricfan.....
     
  2. dannyhw

    dannyhw Senior Member

    Joined:
    Jul 16, 2008
    Messages:
    980
    Likes Received:
    462
    Occupation:
    Software Engineer
    Location:
    New York City Burbs
    First off you need to know how to design the database. It may be a flat 2 dimensional table or it might be a big complex relational database. It depends on the nature of the site. Anyway you need to design, then implement the database in MySQL or something similar.

    Now for the scraping, you want to be able to loop through each page and be able to scrape the relevant information. I like a php library called simpledomparser. It makes things much easier. The firebug extension for Firefox also makes this much easier because you can highlight a page element and it will show you the path to it so you can get an idea of what your code should look like quick.
     
  3. MarketerMac

    MarketerMac Regular Member

    Joined:
    Oct 26, 2009
    Messages:
    247
    Likes Received:
    101
    Preg match in PHP. Google it. Read a lot. Practice even more.

    orrr you can check out the premium imacro's program which is a little simplier to use.