Does anyone have an undetectable selenium jar?

jamie3000

Elite Member
Executive VIP
Jr. VIP
Joined
Jun 30, 2014
Messages
9,440
Reaction score
5,003
Noticing an increasing amount of websites that can detect the default selenium setup now. Probably due to the normal web driver = true check and JavaScript ($cdc + $wdc_) enumeration tricks.

Before I go digging and testing trying to compile my own from source I wanted to just see if anyone here had one they'd be willing to share before I start on my own?

Thanks :-)
 
None that I know of. I think selenium needs these hooks in place in order to work. Are you sure they can be removed?
 
None that I know of. I think selenium needs these hooks in place in order to work. Are you sure they can be removed?

I've seen people reference this

http://www.ifroi.xyz/custom-selenium-jar-file/

However I'm not brave enough to use it lol for obvious reasons.

I've also seen people saying they've managed to get a version of selenium working that passes the youtube detection and distil networks detection, which seem to be the big two. By changing js and webdriver attributes.
 
I've seen people reference this

http://www.ifroi.xyz/custom-selenium-jar-file/

However I'm not brave enough to use it lol for obvious reasons.

I've also seen people saying they've managed to get a version of selenium working that passes the youtube detection and distil networks detection, which seem to be the big two. By changing js and webdriver attributes.

You're right to be skeptical. I wouldn't go anywhere near a modified binary jar like that.

Thanks mate, they go up the ass right?

Yes. Just like regular aspirin.
 
Last edited:
Noticing an increasing amount of websites that can detect the default selenium setup now. Probably due to the normal web driver = true check and JavaScript ($cdc + $wdc_) enumeration tricks.

Before I go digging and testing trying to compile my own from source I wanted to just see if anyone here had one they'd be willing to share before I start on my own?

Thanks :)

Custom Selenium Jar?

I am not sure I have experienced this problem before but haven't done selenium in over a month. What do you mean detect? There is no code run on selenium. Selenium is a library that you interface with a web driver, e.g. chromedriver, firefox (gecko) driver etc.

I don't think this is happening or you misunderstand the reason of why you are detected. If you want give me the website that detects you and I will check to see, I am really curious.
 
Custom Selenium Jar?

I am not sure I have experienced this problem before but haven't done selenium in over a month. What do you mean detect? There is no code run on selenium. Selenium is a library that you interface with a web driver, e.g. chromedriver, firefox (gecko) driver etc.

I don't think this is happening or you misunderstand the reason of why you are detected. If you want give me the website that detects you and I will check to see, I am really curious.

Look up distil networks they use various js tricks to detect selenium browser automation
 
Look up distil networks they use various js tricks to detect selenium browser automation
you said some JS bullshit run, but im sure if you add some randomized actions and do your scraping smarter you can beat their ML algorithm. For these situation false positives are worse than true negatives. So you have the advantage. I don't understand what is there to complain or worry about.
 
any browser can be fingerprinted and so can selenium hence why its a stupid (and bloated) solution to use a headless browser. people do it just because its easy, but its not the way to go.
raw GET/POST requests is what every professional bot uses. the only excuse to use selenium is "because im too lazy to do it properly" :P
 
any browser can be fingerprinted and so can selenium hence why its a stupid (and bloated) solution to use a headless browser. people do it just because its easy, but its not the way to go.
raw GET/POST requests is what every professional bot uses. the only excuse to use selenium is "because im too lazy to do it properly" :p

This forum is so much misinformation. How are you going to load recapatcha or other javascript heavy websites? RAW GET/POST requests? Or you are going to reverse engineer the entire javascript for each site you plan on automating to convert them to raw post/get requests? Everyone is a cheap freeloader here I am sure they will be happy to pay some developer a months worth of salary for doing that. You have no idea what you are talking about or why selenium is used.
 
Hi there, I'm having a similar problem with chromedriver, to describe: the browser instance which is initialized using my python bindings is being detected by Zillow. Yet when I use the chrome browser, launched by my OS gui, Zillow works fine. The captcha on the zillow site, won't even allow me to pass when I complete the challenges manually. After reading this thread, I realize how much more sophisticated my script could be, randomizing the ActionChains and such, but before going into that route, my intuition tells me it's not that complicated. So far I've managed to identify that the default Chrome settings when a ChromeDriver instance is launched are different than the settings when the Chrome Browser is launched from the Favorites bar. I will test out my hypothesis regarding this discrepancy and keep you guys posted. You can see the snapshot of the two instances side by side, ChromeDriver on the Left, Gui Launched on the Right.
 

Attachments

  • Screenshot from 2018-08-01 17-03-36.png
    Screenshot from 2018-08-01 17-03-36.png
    270.6 KB · Views: 91
I tested my hypothesis, and I was wrong. Changing my webdriver chrome settings to mirror the chrome settings of a gui launched chrome instance doesn't allow my webdriver instance to pass the zillow website webdriver detection. I've read other forums which have outlined the DistNetworks problem, but none have mentioned a clear solultion; luckily I am able to get the information I need by automating my browser actions, queries for other websites. BTW the only reason I wanted to scrape the zillow website, via browser, is because the Zillow deepsearch api requires a zipcode input, while the website's search function via the browser is able to return relevant results without a zipcode input. the list of properties I'm querying about has zipcodes, but I determined that some of these were incorrect so I needed a way to return relevant results using just: street address, city, and state. I'm still interested in learning a way to avoid DistNetwork detection, because it seems this is a problem.
 
I tested my hypothesis, and I was wrong. Changing my webdriver chrome settings to mirror the chrome settings of a gui launched chrome instance doesn't allow my webdriver instance to pass the zillow website webdriver detection. I've read other forums which have outlined the DistNetworks problem, but none have mentioned a clear solultion; luckily I am able to get the information I need by automating my browser actions, queries for other websites. BTW the only reason I wanted to scrape the zillow website, via browser, is because the Zillow deepsearch api requires a zipcode input, while the website's search function via the browser is able to return relevant results without a zipcode input. the list of properties I'm querying about has zipcodes, but I determined that some of these were incorrect so I needed a way to return relevant results using just: street address, city, and state. I'm still interested in learning a way to avoid DistNetwork detection, because it seems this is a problem.

I'm also very interested. It's only a matter of time until Facebook, twitter, ig etc all start using similar technology. If you read the white papers distil have they look at over 200 browser properties with js and header analysis. I do believe it's absolutely doable though.
 
Damn I missed this thread. So, did you get the custom web driver compiled by yourself? Subbed :)
 
I have been "digging" around for almost 3 months to find a solution for the Selenium detection. If you track everything up between the default Chrome browser and the webdriver you won't actually find anything different EXCEPT the fact that the Selenium is forcing a few small scripts to be ran in background so that (of course, as always!) google can help itself (catch you) easier on their behalf, where the other sites use pretty much 20-25% of these checks (most of them are using use the distil network which pretty much do the same).
I don't really want to expose all the things that I have found (known reasons) but I can give you a direct tip that I am pretty sure it is going to ring a bell and in 90% of the cases is going to make a difference.

To cut a long story short: The $cdc and the $wdc are the variables (keys) that Selenium use to make checks between the chromedriver and gecko. If you find a way where you can *cough* *cough* *hex* *cough* edit the names of these keys to something else you will make them hell of a problem.
 
I have been "digging" around for almost 3 months to find a solution for the Selenium detection. If you track everything up between the default Chrome browser and the webdriver you won't actually find anything different EXCEPT the fact that the Selenium is forcing a few small scripts to be ran in background so that (of course, as always!) google can help itself (catch you) easier on their behalf, where the other sites use pretty much 20-25% of these checks (most of them are using use the distil network which pretty much do the same).
I don't really want to expose all the things that I have found (known reasons) but I can give you a direct tip that I am pretty sure it is going to ring a bell and in 90% of the cases is going to make a difference.

To cut a long story short: The $cdc and the $wdc are the variables (keys) that Selenium use to make checks between the chromedriver and gecko. If you find a way where you can *cough* *cough* *hex* *cough* edit the names of these keys to something else you will make them hell of a problem.

I also saw a stack overflow post confirming this, but really this shouldn't be openly discussed lol..

Hadn't thought of hex editing, saves the compilation nightmare! Smart thanks.
 
I have been "digging" around for almost 3 months to find a solution for the Selenium detection. If you track everything up between the default Chrome browser and the webdriver you won't actually find anything different EXCEPT the fact that the Selenium is forcing a few small scripts to be ran in background so that (of course, as always!) google can help itself (catch you) easier on their behalf, where the other sites use pretty much 20-25% of these checks (most of them are using use the distil network which pretty much do the same).
I don't really want to expose all the things that I have found (known reasons) but I can give you a direct tip that I am pretty sure it is going to ring a bell and in 90% of the cases is going to make a difference.

To cut a long story short: The $cdc and the $wdc are the variables (keys) that Selenium use to make checks between the chromedriver and gecko. If you find a way where you can *cough* *cough* *hex* *cough* edit the names of these keys to something else you will make them hell of a problem.
Hi, Veil! Did you notice the latest chromedriver version 2.42? It doesn't contain any $wdc_ and, replacing only $cdc_ does not work. I still get blocked by distill. Also tried that with version 2.41 - same sh*t. How do I get rid of distill anti-bot protection? Thanks in advance!
 
Hi, Veil! Did you notice the latest chromedriver version 2.42? It doesn't contain any $wdc_ and, replacing only $cdc_ does not work. I still get blocked by distill. Also tried that with version 2.41 - same sh*t. How do I get rid of distill anti-bot protection? Thanks in advance!

Have not seen it yet. I am working on something lately and once I get my hands on the new driver and see what did they change, I will let you know by posting here. Thanks for the heads up!
 
Have not seen it yet. I am working on something lately and once I get my hands on the new driver and see what did they change, I will let you know by posting here. Thanks for the heads up!
Good, could you tell me on which version of Chromedriver it was possible to do that please?
 
Back
Top