Automation Tool - Scrape, Screenshot, Return.

[Hoco]

Hi,

I'm working on a project and discovered something that may help my prospecting.

I want something that can:

1. plug into Google Sheets (or Excel)
2. copy a specific cell
3. paste into into a service
4. after service has done rendering screenshot the results OR download the PDF generated by the service
5. Store Locally.
6. A bonus would be to copy the result score into Google Sheets.
7. Then repeats 1-6 on the next cell down until blank/no more data.

Would this be achievable, if so what language would be used to build it.

Even better, if you know any libraries that can be used as foundations.

Thanks!

 

[pipitopipita]

Google Apps Script is a language for any kind of automations with G Sheets

 

[Hoco]

Ok thanks.

I need something that is external to G Sheets though that can do the other steps. As far as I'm aware, Google Apps Script won't do it?

 

[CrMas37]

Google Sheet Macros will do your work

 

[Hoco]

Google Sheet Macros will do your work

Including the screenshots and pasting into another service?

 

[CrMas37]

Including the screenshots and pasting into another service?

You can use IFTTT for this

 

[mihai1497]

Instead of using Google app scripts you should try to do it with python.

Even if you don't know python, if you want to start learning to do simple automations python is the way to go.
For getting the info from Sheet/CSV you already have libs for that.
And for using that service you can:
1. Check if they have an API and use that if possible.
2. Use http web request by doing it manually and seeing the request(Chrome/Firefox inspect element go to network and check what request are sent, most of them won't be relevant, like images/js/css, you can also filter).
3. Use a browser automation library like selenium.

 

[DunderMannen]

Scrap the Google sheets shit, it will only automate half of it. As the guy above is saying, try Python.

I think you can do it pretty easily, if you're a newbie, do Selenium instead of Requests to start.

You can automate the sheet part externally in an .xslv file, or even easier a .json file, instead of having to login to google drive with your script too. It'll be more of a hassle imo.

Libraries/Modules;
- Selenium
- OS
- PyScreenshot (?)

 

[Hoco]

Instead of using Google app scripts you should try to do it with python.

Even if you don't know python, if you want to start learning to do simple automations python is the way to go.
For getting the info from Sheet/CSV you already have libs for that.
And for using that service you can:
1. Check if they have an API and use that if possible.
2. Use http web request by doing it manually and seeing the request(Chrome/Firefox inspect element go to network and check what request are sent, most of them won't be relevant, like images/js/css, you can also filter).
3. Use a browser automation library like selenium.

Scrap the Google sheets shit, it will only automate half of it. As the guy above is saying, try Python.

I think you can do it pretty easily, if you're a newbie, do Selenium instead of Requests to start.

You can automate the sheet part externally in an .xslv file, or even easier a .json file, instead of having to login to google drive with your script too. It'll be more of a hassle imo.

Libraries/Modules;
- Selenium
- OS
- PyScreenshot (?)

Thank you both for your responses and modules. I am a complete novice with Python but I can pick up languages easily.

That's what I thought initially via a browser automation (even looked into Macro Extensions on Chromium browsers) - all I need outside of the browser is the data (which is a URL) to be inserted into the service (hence why I though G Sheets would be ideal).

What's really cool as I've just read into Selenium is that I can target html tags too, which would be awesome for an output.

In fact, if so - I'd end up using the outputs as a mail merge - might be able to generate automated emails from this - which would make my job almost redundant... Oops lol.

Is there a collective resource of Libraries/Modules I can scout through?

Are they easy to put together?

I appreciate the advice, I'm more than capable of pulling at the threads if I know where to start .

 

[HoNeYBiRD]

This sounds fairly simple, every each of these processes can be done with iMacros. You don't necessarily need to build a tool for this using any programming language.

Here are a few macros i made before: https://www.blackhatworld.com/seo/i...raper-url-and-email-address-extractor.984618/. They aren't for this exact task obviously, but you need to use pretty much the same commands plus the SCREENSHOT or ONDOWNLOAD command.

 

[DunderMannen]

Is there a collective resource of Libraries/Modules I can scout through?

Are they easy to put together?

I appreciate the advice, I'm more than capable of pulling at the threads if I know where to start .

Well, just look up "modulename + documentation" on google. Alternatively if you dont know what to use; "what you wanna do python library" and you'll find lots of info.

The install process is easy with pip (like npm if you've worked with that). As you may know the syntax of Python couldn't get simpler either, so it's all pretty easy imo.

Best of luck and get to reading

 

[Hoco]

This sounds fairly simple, every each of these processes can be done with iMacros. You don't necessarily need to build a tool for this using any programming language.

Here are a few macros i made before: https://www.blackhatworld.com/seo/i...raper-url-and-email-address-extractor.984618/. They aren't for this exact task obviously, but you need to use pretty much the same commands plus the SCREENSHOT or ONDOWNLOAD command.

This is awesome. I looked into iMacro and GreaseMonkey. The plan was to load them into Firefox to run independently from my main browser. I thought of these as the quick and dirty solution before moving to something like Python. Thanks for pointing me in the right direction (great thread by the way).

Well, just look up "modulename + documentation" on google. Alternatively if you dont know what to use; "what you wanna do python library" and you'll find lots of info.

The install process is easy with pip (like npm if you've worked with that). As you may know the syntax of Python couldn't get simpler either, so it's all pretty easy imo.

Best of luck and get to reading

Brilliant, got a lesson in Google... Haha. Thanks, now I'm more aware that there are lots of libraries out there I'll be certainly putting it all together.

Again, appreciate both of your responses.

 

[DunderMannen]

This is awesome. I looked into iMacro and GreaseMonkey. The plan was to load them into Firefox to run independently from my main browser. I thought of these as the quick and dirty solution before moving to something like Python. Thanks for pointing me in the right direction (great thread by the way).



Brilliant, got a lesson in Google... Haha. Thanks, now I'm more aware that there are lots of libraries out there I'll be certainly putting it all together.

Again, appreciate both of your responses.

Sorry, I think you were looking for a website with it all gathered, nothing vomes up at the top of my head tbh haha. I'm sure there are some w3schools type sites for python too, but someone else has to fill that in for me hehe