How do I scrape databases?

Electricfan

Newbie
Joined
Feb 6, 2009
Messages
3
Reaction score
3
Hi all,
I am trying to scrape some databases to save some time... The pages are coming up with as http://shop.mywebgrocer.com/Browse.aspx?&sid=34474644&sid_guid=f28e0365-7e5e-4e04-8c10-6bd0101b903e&strid=DD39796&catL0=570&catL1=585&catL2=1377&catL3=1015&HasProducts=1


Does this help? It's a site that has grocery store items on it..

Electricfan.....
 
First off you need to know how to design the database. It may be a flat 2 dimensional table or it might be a big complex relational database. It depends on the nature of the site. Anyway you need to design, then implement the database in MySQL or something similar.

Now for the scraping, you want to be able to loop through each page and be able to scrape the relevant information. I like a php library called simpledomparser. It makes things much easier. The firebug extension for Firefox also makes this much easier because you can highlight a page element and it will show you the path to it so you can get an idea of what your code should look like quick.
 
Preg match in PHP. Google it. Read a lot. Practice even more.

orrr you can check out the premium imacro's program which is a little simplier to use.
 
Back
Top