An Open-source bot framework?

J

ju111

Junior Member
Joined
Jul 15, 2011
Messages
144
Reaction score
71
Hi all,

I may take the time to start an open-source bot framework, enabling each one to implement his own bots, and to contribute to the project (for those who wants).

It would ease the process by:

  • Support multi-threads (obviously)
  • rotating proxies
  • be multi-OS (Win/Mac/Linux)
  • easy to use interface
  • have logging and export capabilities in various formats
  • Support Javascript automation through headless browsers
  • enable a relatively easy development of new bots, with the idea to build a full library progressively.
What would you prefer if you would have to contribute and set up a bot?

Language:
  • PHP?
  • Python?
  • Javascript?
  • Other?
I have the feeling that PHP would be easier for people to contribute with new modules/bots, any opinion?

Architecture:
  • Self-hosted web app accessed remotely? (advantage: no need to have a GUI, so you can install the tool on a remote linux server).
  • Standalone graphical application?
Thanks
 
Advertise on BHW
Only Electron fits all your requirements. But it's on nodejs and that's sucks. Code on typescript to be happy.
 
Indeed, Electron is one of the option. But also, a Laravel based application can be developped for instance, fully PHP, with a nice module to interface with headless browsers.

To be more precise: I don't want to know my options, but what would be useful for the community. Many bot-products are expansive, we should be able to help each other through open-source ones. So, I am listening the community.
 
Last edited by a moderator:
Python should be the language for this project. Php is not a suitable language for this.
 
shu1er said:
Only Electron fits all your requirements. But it's on nodejs and that's sucks. Code on typescript to be happy.
Click to expand...

Why do you say it sucks? It's only for a web bot which will have extremely simple commands given to it. And browsers use a lot of JS to begin with. I'm not sure why you'd think nodeJS would suck with Electron in particular since I've never touched nodeJS
 
Hello all,

Finally I selected Python, Django, and purely jQuery. I ll open a IM Journey on this, feel free to bring your opinion and expertise.

Thanks all
 
There's already an open source free framework called Browser Automation Studio. What special will be about your framework besides it being 'another option'?
 
BloodyNinja said:
There's already an open source free framework called Browser Automation Studio. What special will be about your framework besides it being 'another option'?
Click to expand...

It is not open-source. 40$ for 6 months to access all features. Never tested it by the way. Do you use it?
 
In addition it seems to be very windows oriented, with a market place, and so on. Really not open-source....
 
ju111 said:
It is not open-source. 40$ for 6 months to access all features. Never tested it by the way. Do you use it?
Click to expand...

It is open-source. The public repository is being updated with a lag compared to the private one.

github.com/bablosoft/BAS

I am not using it but know a few people who do.

It just looks like you are trying to invent a bicycle :)
 
if any then it would have to be done in C and without headless browsers as they can easily be fingerprinted.
 
BloodyNinja said:
It just looks like you are trying to invent a bicycle :)
Click to expand...

No, not the same philosophy. For me, a click-and-play tools - on Windows - can only lack of flexibility about what can be done. What I need (a personnal need) is a flexible tool to code anything I want without coding multiple times the same features. Of course, I will implement a lot of features which already exist in various products. But it will fit my own needs. I never found a good product to automate things, to scrape, to parse, and to store the data I exactly want in a reusable database. You can point any existing product (and I tested many of them) but... I definitely prefer to learn to do things rather that relying on another product, which will always be at one point a brake to innovation and imagination.

itz_styx said:
if any then it would have to be done in C and without headless browsers as they can easily be fingerprinted.
Click to expand...

Honnestly, I don't think a headless browser can be so easily fingerprinted provided that you can change any header sent to the server... Except (maybe) fingerprinting "webkit" and the version... which should be the same in almost all browsers, but maybe I am wrong on this part.
 
its not (only) about the headers sent, there are more effective ways to fingerprint browsers
 
itz_styx said:
its not (only) about the headers sent, there are more effective ways to fingerprint browsers
Click to expand...

I know, but no one parameter cannot be faked. No need to code in C for that. To move forward in the discusion, let me know which parameters of a browser would enable a) to fingerprint uniquely a browser b) cannot be faked.
 
no "need" to code in C sure, but its way faster as all the scripting languages you want to use for the task. that being said there are many different types of fingerprinting methods and sure they mostly can be faked, but its not always easy.
especially if you use a headless browser that is easy to fingerprint like selenium and all the major ones people use to make it easier for them. the way to go is with direct GET/POST requests and sending the required headers yourself.
 
itz_styx said:
no "need" to code in C sure, but its way faster as all the scripting languages you want to use for the task. that being said there are many different types of fingerprinting methods and sure they mostly can be faked, but its not always easy.
especially if you use a headless browser that is easy to fingerprint like selenium and all the major ones people use to make it easier for them. the way to go is with direct GET/POST requests and sending the required headers yourself.
Click to expand...
So if I wanted to automate creating accounts on a site via headless browser then there is no way to stop the finger printing?

I have been looking into doing an account bot but wasn't sure if I should be trying to finger proof a browser or use headless.


I'm not familiar with GET/POST as much
 
there is always a way i.e. if you heavily modify the code and randomize, but most people just use say selenium as-is. personally i write all my bots without headless browsers using direct HTTP requests.
i mean to make it easier if you aren't much into network programming you can use a headless browser, but for sites like IG and FB i wouldn't suggest that.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top