How to copy Dashboard data to Excel as fast as possible?

ShaolinPower

Registered Member
Joined
Jul 8, 2016
Messages
61
Reaction score
8
Hello everyone. First of all I apologize because I was really not sure where to post this question.

Anyway : I do have access to a huge database of data in some kind of internet dashboard. I would like to copy all the data to an excel sheet but unfortunately, I can't display more than 50 lines of data per page and there is... around 4000 pages of data I would like to copy paste to an excel file.

Currently, this is the way I do it :
- I put my Firefox page at 30% so all data is shown in one page without having to scroll down.
- Select and copy all data
- Control+Tab to quickly get into my Excel Sheet
- Paste all data into Excel Sheet
- - Control+Tab back into Firefox. Select next page.

And repeat for every pages.

Question is : is there potentially any much faster way to do things?
 
Yup.

If you are dashboard is loading the data via API, you can just call that url and store the data. First as json (thats how most APIs transfer data) and then convert them to csv (a format of excel).

How to know if your dashboard is using API?

Open inspector in firefox (ctrl+shift+i probably). Switch to network tab. Reload. You will see a list of networks. One of them is your treasure :P
 
I think the quickest way it's create DOM Crawler on a php or node.js and use guzzle or selenium to imitate browser working and crawl the data to some storage. It allow auth and paginate pages. And just crawl data via DOM of page. But this way good if you need grab data no onetime :)
 
If you want, share the dashboard to my inbox and I can check for loading more data
 
API -> json -> Excel - Fastest
DIY Crawler -> CSV or json -> Excel - Second fastest
 
Have someone write you an excel macro or google sheet script to crawl the page
 
Back
Top