Is anyone into Adversarial Attacks?

Anyone has an interest or played around with adversarial attacks?

Basically fooling/manipulating ai models.

For example:
data poisoning
Adversarial perturbations
language model manipulations
Vision/audio attacks etc

Would be interesting to connect with some of u

Tell us more about it

netizenwizard said:

Tell us more about it

Click to expand...

there is a whole research field in it. But I think it will expand a lot since ai has entered a main stream conciousness.

And in short term I think skills like that will give you an edge bypassing ai detection systems etc
And in long term - if its skynet or totalitarian state largely utilizing some sort of models for control - could be useful to know how to bypass and fool.

But in essence most models etc are very vulnerable to attacks - it just has not been that financially useful to do so.
And their vulnerability is a feature not a bug

That's really interesting, thank you!

Actually I am also interested in finding an ML dev who has experience with CleverHans or anything similar. Adversarial perturbations with noise can be a very nice way to whiten up creatives

Hey,

Currently, I am developing an AI-enhanced fuzzer as a part of my thesis. While working with GPT-4 during my thesis, I engaged in manual prompt injections to bypass ethical guidelines, purely for educational purposes, of course.

In my related work, I explored the paper titled 'GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts'.

I haven't experimented with their project yet, although they do have a GitHub repository. I believe you can download it and experiment on your own. For me, completing my thesis is the priority, but I definitely plan to explore this further.

Additionally, I applied to join OpenAI's Red Team, but unfortunately, the application period had ended. However, I'm hopeful that they will open up applications again. I would suggesst you apply too. Feel free to contact me and let the thread open, if time alllows i will catch up.

Red Team OpenAI:
https://openai.com/blog/red-teaming-network

Source:
https://arxiv.org/pdf/2309.10253v2.pdf

Code:
https://github.com/sherdencooper/GPTFuzz