Custom Website Scrapper

TeKn1qu3z

Senior Member

Jul 26, 2012

1,008

280

• Jun 2, 2017

• #1

I need a custom website scrapper to pull financial data from different websites.

Also needed is a quick way to upload this scraped data back up to my own website.

Maybe the best way is API, need developer thoughts on this. The other way I am thinking about it is storing the data in mysql.

Thanks,

 

B

bananaman5000

Regular Member

Mar 5, 2010

201

130

• Jun 2, 2017

• #2

look up the simple dom html parser, it's a php object and will save you hours of time. it lets you select html elements kind of like jQuery

 

orkney

Newbie

Sep 26, 2016

16

13

• Jun 2, 2017

• #3

I built something similiar using Ubot Studio I built a price checker which scrapes competitor prices daily and imports them into mysql so I can track price changes through a GUI.

 

SEODEMON

Regular Member

Sep 15, 2010

251

78

• Jun 2, 2017

• #4

You need PHP and Curl

 

T

TactSports

Newbie

Jun 1, 2017

36

2

• Jun 2, 2017

• #5

PHP works wonders

 

Dopious

Elite Member

Jr. VIP

Apr 4, 2009

13,911

71,089

• Jun 2, 2017

• #6

Use Diggernaut.com they built a scraper for me and it only cost me $40, you can have their free service or compile the digger and run it from your own computer if you want .. Look them up, I am very happy with their service so far .. Made me a lot of money with my custom built digger at only $40 - It may vary if your digger is more complex in cost, but they are cheap .. =)

Good luck there ..

 

SMMSnab

Registered Member

Mar 30, 2017

91

29

• Jun 2, 2017

• #7

@TeKn1qu3z you can always use webscraper dot io - free chrome extension that scrapes any data you need and export it into csv/xls
then you can import it using sqlizer dot io - convert csv into mysql import script, that can be run in phpMyAdmin