I've been writing bots for around 7 years. I've used a number of languages so hopefully my advice will be useful for you.
Browser Based Bots
Browser based bots are bots that run in a web browser. You use extensions such as iMacros or other tools. I've never had the need or desire to use these types of botting extensions so I can't comment on them.
If you want to do browser based bots I would seriously consider Tampermonkey. You can write bots in JavaScript that utilize the browser. This is a much more powerful way of writing browser bots. I have used Tampermokey a lot and will often sell scripts to clients that use Tampermonkey because they're easy to work with for them.
You can also create native Firefox or Chrome extensions. They work well too depending on the use case.
Headless Browser Bots
A headless browser is a browser that runs in the background. It's invisible. You program it to perform actions on a website. For example, if you write a bot to post status updates on Facebook using a headless browser, when you run the bot it will literally navigate to Facebook and perform the required actions to login. You just don't see it.
I used a headless browser to create a Facebook Ad Scraper for a client. It worked out very well.
Here's a couple of popular headless browsers:
- Selenium
- Webkit
- PhantomJS
- CasperJS
- NightmareJS
There's more ... but those are the ones I use.
HTTP Bots
A http bot can be written in any language that can send HTTP requests. Everything that you do on the Internet is done with HTTP requests. When you enter your login details on Facebook and press "login" it sends a HTTP POST request to Facebook with the required data to perform the login.
When writing HTTP bots you have to clone this request and rewrite it in your language of choice passing in the required data (username, password, a csrf token if one is required).
HTTP bots are my favourite. People think that websites that use Ajax are secure from these types of bots. That's incorrect because you just make a HTTP request to the Ajax endpoint. Easy.
Things can get tricky though. For example a site that relies on JavaScript can seem impossible to bot. I once did a client job on a site that set the CSRF token with JavaScript. The method for generating the token was obfuscated hidden deep in their JavaScript. It relied on other data to generate the token. I basically had to replicate these methods in my bot to generate the correct token.
Recommended Languages
- Python
- NodeJS
- Bash
- C#
- JavaScript
- Go
- C/C++
Scripts vs. Desktop Applications
I started writing bots using .NET. I would create complete desktop applications with lots of features and buttons on the GUI.
Later I ditched Windows for Linux and could no longer create bots with a GUI unless I stuck to using Windows, or invested some time into using Qt. I had to find another way.
I started scripting. Which means writing small scripts or programs that can be ran on the terminal. After a while of doing this, and learning Linux, I felt like I just levelled up into a completely new level of bot writing.
Some bots could be written in pure bash utilising Linux programs such as cURL, grep, sed and awk. Seriously, you can write some powerful bots in Bash alone. What I loved about Linux and scripting was I could put things into cron jobs, or pipe the output of one bot into the input of another. You can also utilise databases such as Redis, or message queues like ZeroMQ much easier.
The Unix philosophy is true. Write lots of small programs and together they make a large application. Which is what I do now. I will script a lot of small utilities in NodeJS, Python or C and chain them together in any fashion I like.
The benefit of doing it this way means I can spin up a server on Amazon or Digital Ocean, pull in my scripts from my repo's and hey presto I now have a dedicated box running my bots 24/7
I rarely create applications with a GUI unless they're intended for commercial use. The development time takes much longer, and when it comes to putting it on a server and utilising automation things get costly and more of a challenge.
I hope the advice has been useful. If you have any questions at all feel free to ask.