Guys, I have 2 jobs that i need to do urgently. 1) I need to scrape data from a web site - this one im okay with but i need to decide which tool to use? uBot or Zenno? Any advice on the best one to use? 2) This job is the tricky one? similar to to the first job but its not a web based product i need to scrape but a piece of software? I have no idea how i can start to scrape the results of desktop software? Suggestions? if its even possible?
How does this desktop app work? You can't scrape a desktop app directly, you'd probably need to intercept its network data....
the desktop app is a little like excel .... you perform a search and the data is returned in columns/rows. I need to be able to perform the search and collect the data?
it can be done but you need some expertise with win32 commands. Basically you use sendkeys to send search and copy commands to the desktop software. The result returned is copied to clipboard and you get the data.
Windows automation software. Have a search on Google and pick one. No idea if there are any free ones available. If you can code I'd recommend Perl with Win32::OLE module.
It depends. If the database of the software is stored locally, best way would be to decode it. If the database is stored on remote server you'll need to figure how the program makes a request to the remote server. Then perform a bunch of requests to retrieve everything you need.
As usual you post about things you have no idea about. There is no interception involved. It is a case of automating the GUI.
http://www.demondemon.com/2014/04/11/meta-data-or-big-data-extraction-from-the-web/ This and excel scrape just about anything. Images, CSS, tables, lists. Software like this can be programmed to find links, click to "Next page", follow links to a depth, follow links of a certain format - scrape whatever it finds down there, decent file control.. blah blah. Free trial so probably worth a look (I don't sell it - I just use it) Wrote something like this myself in Python, then realised this was a ton better - well this and a quick export to Excel or Openoffice Calc and ten minutes fiddling and filtering. Scritty
