1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How do I get started on web automation?

Discussion in 'General Programming Chat' started by Descension, May 18, 2016.

  1. Descension

    Descension Newbie

    Joined:
    May 3, 2015
    Messages:
    3
    Likes Received:
    0
    Hello!


    I want to get into learning web automation through a Linux Server

    I would like to be able to test on my windows computer and be able to run it on my Linux server.

    When I say web automation, I mean like being able to click on things on websites, like a normal browser.


    I have been reading a bit about Selenium and that it might be useful for headless web browsing.

    I know java programming and I am willing to learn a new language.


    What is the best way to do web automation from a Linux server?

    What tools can I use?

    What programming language is required to know for the most effective web automation?


    Thank you for your time!
     
  2. tvm77

    tvm77 Jr. VIP Jr. VIP

    Joined:
    Nov 14, 2013
    Messages:
    169
    Likes Received:
    68
    Home Page:
    Hi,
    "When I say web automation, I mean like being able to click on things on websites, like a normal browser."
    a linux server don't run any kind of web browser so you can browse or click with a software, the best you can do in a linux server is send http post / get call to urls but will not be very effective
     
  3. Descension

    Descension Newbie

    Joined:
    May 3, 2015
    Messages:
    3
    Likes Received:
    0
    I was thinking about headless browsing. Is that not possible on a linux server? I know you won't be able to see the clicking but I could imagine it would still do it.

    Do you think selenium could work on linux?
     
  4. MrBlue

    MrBlue Senior Member

    Joined:
    Dec 18, 2009
    Messages:
    969
    Likes Received:
    678
    Occupation:
    Web/Bot Developer
    LMAO. It's funny how people who have absolutely no idea are always the first to give advice.

    There are many great ways to implement web automation from a Linux box and using Selenium with Java is certainly one option for both normal and headless browsers. There is also CasperJS (http://casperjs.org/) which is a navigation scripting & testing utility written in JavaScript. You can find a full list of available headless browsers along with their supported language here: https://github.com/dhamaniasad/HeadlessBrowsers.
     
    • Thanks Thanks x 2
  5. kahuna74

    kahuna74 Regular Member

    Joined:
    Aug 19, 2014
    Messages:
    270
    Likes Received:
    102
    Gender:
    Male
    Occupation:
    Software Developer
    Location:
    Grand Rapids, MI
    Like MrBlue said... Casperjs or Java/Selenium. I have had good luck with both. casper is easier but perhaps not quite as flexible as something driving selenium.
     
    • Thanks Thanks x 2
  6. starfish89

    starfish89 Newbie

    Joined:
    Sep 4, 2013
    Messages:
    5
    Likes Received:
    3
    Java + Selenium is the way to go :)
    You can also do headless automation wiht selenium on Linux.
     
  7. Iamhere123

    Iamhere123 Newbie

    Joined:
    Jul 28, 2017
    Messages:
    13
    Likes Received:
    0
    Gender:
    Male
    Linux is the best way to bot period. So much more functionality then windows. And yeap, firefox + chrome headless sounds amazing. Can't wait to start using them. If only I could work out how to scrape pages with Selenium *shrugs*. Oh well I suppose I'll look at 500+ pages on stackoverflow and then realise I was over complicating things... xD
     
  8. Sergey Yermak

    Sergey Yermak Newbie

    Joined:
    Mar 12, 2016
    Messages:
    29
    Likes Received:
    2
    Gender:
    Male
    Occupation:
    Node.js
    I'm on linux and for now using Node.js for this. There are many different modules you can use. For maximum user imitation you can of course use selenium, or other headless like phantomJs.
    But if you want more speed and scrape large amount of data it would be too slow. Better to use raw requests then.