Selenium Bot Question - Time to Create

sirmeep

sirmeep

Registered Member
Joined
Jan 16, 2017
Messages
98
Reaction score
44
Question for those of you who build bots out of Selenium.

How long does it takes you code up a site specific bot. For instance one that creates an account and performs a single function (like login to facebook and post on a wall). Understand that there are a lot of variables, but trying to understand the order of magnitude. Is it like 15minutes, 1 Hour, 3 Hours?

I'm currently using PYAUTOGUI, and thinking of spending a few days to pick up selenium. Trying to see if the time savings are worth the extra invested effort.
 
Advertise on BHW
If there is no other obstacle like captcha or worrying about proxy, then login bot is 5 minutes. Posting is another 5. Definitely worth it IMO.
 
Selenium for Python is really easy and simple. Just locate elements you want to click on and after that it's really easy. To answer your question, it depends. If you simply want to login to somewhere it can literally take 1 minute if you already know Selenium.
 
Anyone know what's wrong with selenium ide for firefox?
 
Ahh Ok. Thanks for the feedback. It's taking me around an hour per site to build a bot right now. So it's worth the effort to switch.
 
Just watch out if you are using Selenium with Chrome, as I read somewhere that Google sends data to register a running Selenium driver.
 
Redaline said:
Just watch out if you are using Selenium with Chrome, as I read somewhere that Google sends data to register a running Selenium driver.
Click to expand...

Yes, you are right. But you can disable this with some plugins
 
For a simple scraping bot it takes around 30 minutes but it depends on how complex the project needs to be.. e.g. database tables, multi threading and so on. A project which has db and multi threading generally takes around 3-5 hrs.
 
The problem with selenium is that it can easily be detected specially if you are running chrome.

In terms of captcha Google can tell you're using selenium but you can complete them (but only the images), and only so many requests until you get the message "You network is sending automated requests".

I tried using selenium to pull the Voice verification then running it through Google Speech to text. Worked like a charm until I fired up selenium and it gave the the Network is running requests haha.

There is a way you can stop that from happening I did see someone on GitHub that was able to interfere something (I just don't exactly remember).

You have to remember that fingerprinting is a big deal. This means aged fingerprints is the best (fingerprints with a track record).

In terms of making a bot with selenium it's rather easy just do .findElementbyXPath and use ChroPath to find the XPaths on Chrome - this is the fastest and easiest way.

Very simple stuff honestly.

Depending on the bot it can take a simple 15 minutes, to 1 hour (if they have captcha for example or if they use alot of HFO html field obfuscation)
 
Agreed with Digital Finger, fingerprinting is quite important.
Another thing - you may want to modify the chromedriver source code and replace some unique variables with different names and values..

On time to code: on average up to 1h for simple task. 30 minutes for simple sites, 2++ hours for sites who proactively fight the automation.

For google recaptcha etc, look for human powered anti captcha solutions, some are quite cheap and stable.
 
Andy17 said:
Agreed with Digital Finger, fingerprinting is quite important.
Another thing - you may want to modify the chromedriver source code and replace some unique variables with different names and values..

On time to code: on average up to 1h for simple task. 30 minutes for simple sites, 2++ hours for sites who proactively fight the automation.

For google recaptcha etc, look for human powered anti captcha solutions, some are quite cheap and stable.
Click to expand...
Yeah i read that theard on stackoverflow too bad it dosent work anymore
 
if its just login it can be done in 30-45 minutes, 90% of that time goes to finding out paths/classes
+ error handling

and for like adding a simple function after logging in the time taken will depend on the fucntion

for me, I built an amazon product scraper
which took me 4-5days

if you have older scripts similer to that you can easly copy paste and change it according to your needs it saves some time
 
My 5 cents. To organize multi-threaded bot work, I use Firefox and Chrome portable browsers.
Each thread works with its own browser settings (proxies, UA, plugins, etc.).
The maximum that I was able to run on a computer with 16GB RAM - 50 simultaneously opened browsers, in which there was a posting to Facebook groups on behalf of 50 different Facebook accounts.
Therefore, I recommend using portable browsers to work with selenium.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top