Attempts to solve arkoselab funcaptcha with AI.

[dohyung97022]

Buongiorno everyone.
This thread is for sharing my journey to solve arkoselab funcaptcha with self created AI.

Why would I want to do this?

You might be doubting this idea because the captcha is specifically designed to distinguish AI with a actual human being.
Well most assumptions of this doubt is correct, but...

1. There are already attempts in the captcha solving industry to bypass with AI.
You may have not heard of such service, since it is not mentioned AT ALL in this forum. and I do not know why...?
I have been using this service for a while, until I have hit a hard limit.
This service cannot solve the most recent captchas created by arkoselab.
Microsoft bing has updated their captchas to the newest version of arkoselab.
So all the automation I was doing has been stopped. Xd
That is why I want to make my own solution.

2. I have my fair share of experience using captcha solving services and methods.
although I have not used all of them in my projects, I have used some of them to know enough on how they function.
So you can believe in my words. (if you want of course)
I have also solved a few captchas with my own ai model.

I am a self trained profession.
I do not know what I am doing. Do not trust my word for it.
Do not trust anything in this thread. And most importantly do not follow this at home. XD

3. Most captcha solving services take too much time.
15 seconds for a solve?
Some captchas would ask you to solve it again if it takes too much time.
the ai service was a bit faster than the other methods,
but still it had problems when sometimes the ai would not respond at all,
I assume this is due that the ai could not understand an image if it is a bit complicated than others.

Why would I want to share this?
I am a lazy little pig. If no one watches me, I get lazy by the day.
I will share some of the knowledge I feel like sharing,
but I will not share everything. Only ideas and concepts.

I do not sell anything, I will do nothing for you.
I cannot solve your life problems.
You are on your own. (if you did not know that already)

Enough of this nonsense gibberish,
So what am I going to solve?
Arkoselab funcaptcha v1.


You can see this captcha in the registration in microsoft bing.
I do not know how they call this captcha, but I will call this funcaptcha v1.
I also know that arkoselabs give different captchas according to utc timeline.
I have seen captchas asking to pick a sphere made out of grass. (Some of you might have seen this...)
But for now I will solve this one asking for the image with the same icon.

How am I going to solve this?
Well thats what I will be sharing.

How will I post?
I will be posting 1 post per day.
I will try to keep post quality as great as possible. (no pinky promises)
If this seems to be solved already, I will conclude this thread.
If things are not working out, I will post out a failure report on my thoughts on why it does not work and retry the process.
If things does not work out even with 3 or 4 retries? I will give up and conclude this thread.
If all things fail, It is still not the end of the world for me.
We still have the option to go manual. XD


How do you know I am serious and this is not a made up story by a 10 years old?
My previous thread. [Twitter automation test (4 factors, 32 accounts)]
My servers running automation.

Wish I could say that they were connected with kubernetes klusters, but unfortunately they are not. (Yet)
I just have shell files that install my api and run automation.
I also have my own proxy server with lte ip addresses.
I wish I could share this with you guys, on how I have done this, internal networking and all that fun stuff.
But I will keep that joy for another day.
So yeah, I am dead serious.

And most importantly?
Lets go loco and have fun.

 

[Proxy Cake]

This is really interesting, there will be great opportunities to sell your service or access to your API on the forum if all goes well.

Good luck!

 

[dohyung97022]

This is really interesting, there will be great opportunities to sell your service or access to your API on the forum if all goes well.

Well, not selling anything on this forum.
If I sell it, It would be compromised and abused by everyone and soon to be patched.

I think the real money would be in social media itself.
That is at least what I believe. (for now)

But thanks anyways XD

 

[PrettyToughJob]

Cool. You are really spending your resources for this. Good luck mate!

 

[FatBee]

All right, let me break it to you. Solving the images is the easy part. You need to reverse engineer the BDA and replicate it perfectly every time, then solve the images.
BDA is the fingerprinting for you/your browser, which essentially gets scored after submitting the solution.

A solved captcha is not a good captcha. Remember that.

Also, funcatpcha made it "easy," but the harder you get fingerprinted, or the more bot-like you are, the scale of the challenges is turned in to 0 chance of solving with AI.

Don't ask me how I know :v

 

[dohyung97022]

All right, let me break it to you. Solving the images is the easy part. You need to reverse engineer the BDA and replicate it perfectly every time, then solve the images.
BDA is the fingerprinting for you/your browser, which essentially gets scored after submitting the solution.

A solved captcha is not a good captcha. Remember that.

Also, funcatpcha made it "easy," but the harder you get fingerprinted, or the more bot-like you are, the scale of the challenges is turned in to 0 chance of solving with AI.

Don't ask me how I know :v

Thanks for the insight.

I did useragent spoofing which seems to be good enough when I was automating bing.
But I am not that professional enough to change web gl or other fingerprints.
And also I did suspect that they change the ai challenge question to a harder one if the same fingerprint was used again and again.
I had challenges change to a harder one on bing and on twitter when solved multiple times.

Arkose labs actually state this on their site.


I thought of using a common browser with a common os and a common graphics card like windows 10 with chrome.
Did not test that thought yet.
Wish I would find a better way, I would have to change all my servers to windows 10...
I would hate that...
And I also heard that some people use android to bypass captcha.

Can we share secrets? XD

 

[FatBee]

Thanks for the insight.

I did useragent spoofing which seems to be good enough when I was automating bing.
But I am not that professional enough to change web gl or other fingerprints.
And also I did suspect that they change the ai challenge question to a harder one if the same fingerprint was used again and again.
I had challenges change to a harder one on bing and on twitter when solved multiple times.

Arkose labs actually state this on their site.
View attachment 245599

I thought of using a common browser with a common os and a common graphics card like windows 10 with chrome.
Did not test that thought yet.
Wish I would find a better way, I would have to change all my servers to windows 10...
I would hate that...
And I also heard that some people use android to bypass captcha.

Can we share secrets? XD

There is no secrets top be honest, mobile bypassing for funcap is possible when devs fuck the integration and you can simply pass by it

 

[dohyung97022]

There is no secrets top be honest, mobile bypassing for funcap is possible when devs fuck the integration and you can simply pass by it

You are right.

Was interested in your topic and saw a awesome thread,
https://www.blackhatworld.com/seo/spoofing-canvas-and-webgl.1233348/
Some did claim that they can spoof js functions or intercept requests of fingerprints before they are sent to the website for varification.

Request interception could be possible with selenium-wire.
https://pypi.org/project/selenium-wire/#intercepting-requests-and-responses
But I do know for a fact that selenium-wire is detected by TLS fingerprinting.
https://github.com/wkeeling/selenium-wire/issues/427
They could have also hashed the fingerprint with a different unique key every time the request is sent.
Which would not be any good if used again and again, because they would detect that by key.

So, I guess my best bet would be with android, or ios device, computers with common hardware, as you mentioned.
Bing shows only one challenge question on register by android, so this looks promising for now.
Don't know if its still the case, when repeated multiple times.

Thanks.

 

[dohyung97022]

Report Day 1

I did some researching for ai algorithms,
I will share with you the overal concepts before we begin.


What is the question?


This captcha is asking 2 simple questions, and some subquestions that goes along with it.

Q1. Can you detect where the icon is?
Q1-S1. Can you detect where the icon is despite the background?
Q2. Can you tell that the icon is same?
Q2-S1. Can you tell that it is the same despite its difference in scale?
Q2-S2. Can you tell that it is the same despite its difference in rotation?
Q2-S3. Can you tell that it is the same despite its difference in opacity?


Q1 Can you detect where the icon is?
Detecting where the icon is (Q1) is fairly simple.
We can just use any detection algorithms out there and with enough data, it would work fine.
Common detection algorithms would return us the x1,y1,x2,y2 cordinates (box) of where the object is.

Common detection algorithms.



Q1-S1 Can you detect where the icon is despite the background?
Detecting the icon despite the background is also simple.
Modern algorithms also provide segmentation detection.
You can detect the exact pixcels where the object is.

Segmentation detection.


So we could detect the icon pixels.
Remove all other background pixels that are around it.

The problem is when the segmentation ai is too complex of a task for the ai to handle.
We can always divide the task into parts for a different model to handle.

So my first plan would be,



And if this is too hard, I can add a additional step that finds the location.



Some modern models only handle segmentation, some handle boxing and segmentation at the same time.
But the theory is that if we devide the tasks into more simple ones,
the ai would have a much better time understanding what the f(x) would be.


Q2 Can you tell that the icon is same?
Now this could be a harder task for us to solve.
A lot of the ai algorithms out there is mostly for classification.
Is this a dog, is this a cat and so on.

So if we go through the traditional classification approach,
We would have to go through all the icons and classify each one of them.
This would take forever to classify.
If somebody asks me to replicate this captcha, I would use a site like font awesome.
which has thousands and thousands of icons that they use.
I would have to label thousands of classes.
And the worst part would be that I would have like 5 or lesser images for each class.

So the traditional classification would be a big no no.

I have been studying on how ai works, and was supprised that it was quite simple.
Sendex AI tutorials. (Big fan btw)
https://www.youtube.com/playlist?list=PLQVvvaa0QuDfhTox0AjmQ6tvTgMBZBEXN
So how I understand AI, is that it is a f() in the f(x) = y.
We know the x, and we know the y, so we keep changing random variables in f() until it keeps giving the right y.

Very simple.
So back to our question,

Can AI tell icon 1 is same as icon 2?

so our function would be,
function ( icon_image_1, icon_image_2 ) = is_same

We need to give 2 icons in order to create our function.
There are already studys and papers on this approach about multiple image inputs.

This is also a simple example of multiple input AI models in youtube.

This guys function would be
function ( poketmon_stat, poketmon_image ) = poketmon_health

There are quite a lot of papers that shows promising results in this,
so we can definitely solve this with the multiple input approach.

Q2-S1. Can you tell that it is the same despite its difference in scale?
The Icons width and height portion would be the same.
We have the boundary box, and if we change both icon images into a same width and height, the icons would be identical.


Q2-S2. Can you tell that it is the same despite its difference in rotation?
Well I have not come up with a great idea on this one.
I am a bad programmer Xd, so I will brute force it.
Which means I will rotate it 1 degrees and run it 360 times. XD

Some may say that this could be done by ai,
And you are true, there are some cases that has solved the rotation problem in ai.
https://developer.nvidia.com/blog/detecting-rotated-objects-using-the-odtk/
But the problem would be that,
how do we define a rotation of a icon?
If there is a icon of a door, we know that it has to be vertical,
but a door being vertical is known by common knowledge not by looking at multible icons.

We could create an ai, and hope that it spits out the same rotation. (possible)

function (icon_x1) = straight_icon_y1
function (very_simmilar_icon_x2) = very_simmilar_straight_icon_y2

So if it spits out the same degrees of rotation, we could use that image to compare.
But this would add another layer of complexity.
Brute force ( 0 ~ 360 degrees ) still feels much more accurate to me,
so you will have to trust me on this one.


Q2-S3. Can you tell that it is the same despite its opacity?
We can modify the image to be grayscale and use image modifying methods before we feed it to the ai.

There are tons of image modification methods and tutorials out there,
did quite a lot of research myself,
I think sendex is best.
https://www.youtube.com/playlist?list=PLQVvvaa0QuDdttJXlLtAJxJetJcqmqlQq

What did I do today?
I created a scraper that scrapes funcaptcha images, and crops them into 6 segments.
I will try to mask label the images tomorrow,
I have life stuff, so don't count on it too much.
Thursday is quite loose, so I wish a few hundread would be done by them.

Thanks for reading this shit long post.
Anyways see you then.

 

[dohyung97022]

This will not be continued.
Guess others have solved this problem already.
A bit of wasted time... but learned a thing or two... Xd

 

[th3darkknight]

This will not be continued.
Guess others have solved this problem already.
A bit of wasted time... but learned a thing or two... Xd

Crap, I was reading the whole thread with a lot of interest. But it's discontinued :3

Can you let me know how to achieve these results?
Thank you.

 

[dohyung97022]

Crap, I was reading the whole thread with a lot of interest. But it's discontinued :3

Can you let me know how to achieve these results?
Thank you.

Step 1. Any rcnn AI.
Step 2. Image segmentation.
Step 3. Python Pillow, or OpenCV to crop the segmentation.

More about Image segmentation.

 

[lotsofmoney]

You look like expert in automation. Where you learnt all this?

 

[dohyung97022]

You look like expert in automation. Where you learnt all this?

Mostly here, youtube, and stackoverflow.