GPT Detectors Don't Work: Source? OpenAI

malayan

malayan

Power Member
Joined
Aug 26, 2020
Messages
608
Reaction score
554
Early today OpenAI posted guidance for educators using GPT in the classroom and one of the subjects they covered was "detecting" GPT content. For all of the people out there worrying about Google or any other site classifying and demarking your content as AI, this is for you.

Many of these insights might have been intuitive, but now you can see--straight from the horse's mouth:

https://help.openai.com/en/articles...-presenting-ai-generated-content-as-their-own
Do AI detectors work?
  • In short, no. While some (including OpenAI) have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.
  • Additionally, ChatGPT has no “knowledge” of what content could be AI-generated. It will sometimes make up responses to questions like “did you write this [essay]?” or “could this have been written by AI?” These responses are random and have no basis in fact.
  • To elaborate on our research into the shortcomings of detectors, one of our key findings was that these tools sometimes suggest that human-written content was generated by AI.
    • When we at OpenAI tried to train an AI-generated content detector, we found that it labeled human-written text like Shakespeare and the Declaration of Independence as AI-generated.
    • There were also indications that it could disproportionately impact students who had learned or were learning English as a second language and students whose writing was particularly formulaic or concise.
  • Even if these tools could accurately identify AI-generated content (which they cannot yet), students can make small edits to evade detection.
Click to expand...
The wider-reaching implication from this is: GPT has (unsurprisingly) been unable to develop a watermark for detecting AI content.
 
Advertise on BHW
OpenAI stopping access their own detector tool back in July because of "its low rate of accuracy" was the first nail in the coffin for this snake oil business.
Additionally, ChatGPT has no “knowledge” of what content could be AI-generated. It will sometimes make up responses to questions like “did you write this [essay]?” or “could this have been written by AI?” These responses are random and have no basis in fact.
Click to expand...
Imagine if that was the entire workflow of some of these tools - to find out whether this text has been written by ChatGPT, I'll just ask it! - lol!
 
I tried removing all 3.5 patterns but finally I realized that AI is trained from human texts.
Even though it's obviously AI pattern like "it's important to note that", "However" with AI disclaimers but the sentence structures are indeed used by real human writers.

Now I only manually edit some of the most obvious AI structures. For the small AI patterns, I keep it as is.

The AI detection business is going crazy trying to milk out the average users. IMO as long the obvious AI writing structure is taken care of, the content can be treated as human content.

The small footprints are just marketing hype.
 
I tested several tools text and and promps and had multiple AI detectors until I notice the same, they just don't work and I was wasting my time, chatgpt dabatase is from information from the internet that is feed from their engineers, it means that the information that you ask to chat gpt is not created from thing air, is information from the internet!!! so why AI detectors detect internet information as AI if it was already there? after thinking this I just stopped using those AI detectors and also that AI is trained to look human so there isnot tool that can really say this is AI 100% all the time.
 
All those AI content detectors are just milking the market
Zero utility imo
 
stopped caring after putting a few - :- : into article it went from 100% ai to 100% human
 
4440 said:
I tried removing all 3.5 patterns but finally I realized that AI is trained from human texts.
Even though it's obviously AI pattern like "it's important to note that", "However" with AI disclaimers but the sentence structures are indeed used by real human writers.

Now I only manually edit some of the most obvious AI structures. For the small AI patterns, I keep it as is.

The AI detection business is going crazy trying to milk out the average users. IMO as long the obvious AI writing structure is taken care of, the content can be treated as human content.

The small footprints are just marketing hype.
Click to expand...
It's all marketing hype, but people want to believe that human output is special--so they need detectors to differentiate it from the GPT work.

The truth is GPT can get as nuanced and creative as 99% of writers alive.
 
After they recognized my text as AI, I doubt that they have any sense
 
A client of mine once showed me that my content was AI detected. I asked him the website and it was detecting everything as AI written. Since most of my content is referenced from some online source and rewritten accordingly. The Tool was detecting it as AI written assuming the content was rephrased by an AI. I recorded and showed the same thing to my client, he was surprised to believe it. After that he never complained.
 
alonelywriter said:
A client of mine once showed me that my content was AI detected. I asked him the website and it was detecting everything as AI written. Since most of my content is referenced from some online source and rewritten accordingly. The Tool was detecting it as AI written assuming the content was rephrased by an AI. I recorded and showed the same thing to my client, he was surprised to believe it. After that he never complained.
Click to expand...
I checked more than 20 detector sites. Everyone's results are different. Almost always, detectors check the length of the string and draw a conclusion from this.
 
AI detectors simply don't work. Originality AI was one that everyone was recommending but i still found it not to be accurate
 
Until AI detectors can be true to themselves, no one can be held responsible for outsourcing more information online. Most of the hyped detectors are less accurate at detecting AI-generated text that is long, complex, and original.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top