OpenAI Shuts Down AI Detection Tool
Why It’s Difficult to Distinguish Between Real and AI Content
Did you know that, by 2026, experts predict that about 90% of online content will be AI generated? You may immediately feel uneasy about this prospect, but, the truth is, much of this content will be indistinguishable from human-written work. In fact, it’s become almost impossible to differentiate between human and AI content, as OpenAI, the creators of ChatGPT, found with their classifier tool.
The Launch of OpenAI’s AI Classifier
After the quick rise of AI models in late 2022, and the subsequent boom earlier this year, OpenAI launched an AI classification tool, aimed at distinguishing between real and AI content. To do this, OpenAI fed an AI classification model samples of human and AI-written content, and tasked it with categorizing them into related categories. The idea being that the more training data the classifier correctly categorizes, the better able it is to identify AI content.
However, in its initial announcement post, OpenAI caveated the tool:
“Our classifier is not fully reliable. In our evaluations on a ‘challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases.”
OpenAI was transparent about the initial lack of accuracy the classifier achieved. Still, the hope was that it would improve with ongoing training and development. Unfortunately, as you can tell from the title of this post, the detection tool experienced no significant improvements.
Shutting Down OpenAI’s Detection Tool
While an official announcement has not yet been made, OpenAI released an addendum to their original blog post, stating:
“As of July 20, 2023, the AI classifier is no longer available due to its low rate of accuracy. We are working to incorporate feedback and are currently researching more effective provenance techniques for text, and have made a commitment to develop and deploy mechanisms that enable users to understand if audio or visual content is AI-generated.”
Interestingly, the classifier was discontinued in late July, but many users are only recently noticing its absence, highlighting the lack of interest surrounding the tool. In fact, in this week alone, several articles have been published discussing its closure.
OpenAI’s detection tool being discontinued poses several questions and concerns. For example, will we ever be able to accurately identify AI content if even the creators of ChatGPT cannot? By solely relying on probabilities, statistics, and computer-led content comparisons, it’s highly unlikely. As a result, properly identifying AI content may be a job better suited for humans.
Why Is AI Content so Difficult to Detect?
Simply put, identifying AI-written content is challenging because it’s not actually meant to be distinguishable from human text. AI generators, such as ChatGPT, Claude, and Bard, are made to closely replicate human content, and have been trained on large volumes of human content to allow them to do so.
Now, you’re probably thinking that you can, in fact, tell the difference between AI content and human content. It’s true that, despite how far it’s come, most AI-produced content is still lacking in quality. Unfortunately, AI content detectors don’t take quality into account; instead, they work based on averages, probability, and comparison. This means they can sometimes identify human content as AI generated only because it hits on a number of arbitrary statistical parameters.
How Can You Identify AI-Written Content?
Although OpenAI’s classification tool is ineffectual when identifying AI content, it is possible to identify AI content yourself by looking out for several things. This includes:
Contextual Understanding: AI has come leaps and bounds in terms of mimicking the human understanding of context. However, AI text generators often still fall short of a comprehensive understanding, especially when dealing with complex concepts. As a result, if you suspect that someone has used AI to generate text content, take note of their apparent understanding of context. If their content is obviously misdirected, then they may have used artificial intelligence.
Spelling and Grammar: No matter how professional the writer, humans are prone to making mistakes, particularly in regards to spelling and grammar; yet, we still try. AI-generated content, on the other hand, is often devoid of proper grammar or exhibits an overuse. For instance, you might find AI content contains an absurd abundance of exclamation marks. Alternatively, you may also discover instances of inconsistent capitalization or punctuation use.
Fragmented Sentences: A unique quirk of AI text content generators is that they sometimes stop in the middle of sentences. This is a huge indicator that content is AI generated. While humans might forget the odd comma or two, most writers don’t leave sentences hanging, and then move onto the next.
Formatting Issues: If a writer copies content from an AI generator into a Word or Google doc, you may notice some strange formatting issues. For example, font and text size may be different to the default, while numbered lists and bullet points might not appear properly indented.
Of course, the biggest issue with attempting to identify AI content yourself is that you might be wrong. Without hard data to back you up, there is no way of truly verifying that a piece of content is AI generated. And that’s the rub: AI generators are now so advanced and AI content detectors so arbitrary, that it’s really impossible to know for sure whether content is real or not. This is a living nightmare for authentic writers and businesses alike.
What Tools Can Help?
While it’s impossible for an AI content detector to accurately establish whether content is AI or human, there are a handful of tools that can help. Using these tools in combination with the above knowledge will help you make an informed judgment about the authenticity of content.
Copyleaks: Copyleaks is an AI content detector like OpenAI’s classifier, but it is generally regarded as one of the better tools available. In fact, Copyleaks is endorsed by Semrush, Medium, and Edmentum. Nevertheless, the results of this tool should still be taken with a pinch of salt.
Really Wrote It: Really Wrote It takes a different approach. Instead of attempting to estimate AI content probability after the fact, it asks writers to create content within their anti-AI word processor. The platform tracks each keystroke, allowing writers to playback their writing process and easily prove authenticity.
AI Business Report: That’s right, by reading this blog, you’re on your way to identifying AI-generated content. Why? Because the better you understand how artificial intelligence works, the better you’ll be able to spot the signs.
Thanks for reading.
If you enjoyed this article, please subscribe to receive email notifications whenever we post.
AI Business Report is brought to you by Californian development agency, Idea Maker.
Sources: