OpenAI launches new tool to detect AI-generated text

OpenAI, an American artificial intelligence (AI) research laboratory and the company behind ChatGPT, has launched an AI Text Classifier that attempts to distinguish between human-written and AI-generated text. 

OpenAI captured the public’s attention when it introduced ChatGPT in November last year, which is a chatbot that generates text that might seem to have been written by a person in response to a person’s prompt. The popularity of ChatGPT since its launch has given rise to authorship concerns as students and workers use the bot to create reports and content and pass it off as their own.

The new "AI Text Classifier," as the company calls it, is a "fine-tuned Generative Pre-trained Transformer (GPT) model that predicts how likely it is that a piece of text was generated by AI from a variety of sources," OpenAI said in a blog post.  

To train the text classifier model, OpenAI used human-written text from a Wikipedia dataset, a 2019 WebText dataset and human demonstrations that were used to train InstructGPT, another language model. The company said it used "balanced batches that contain equal proportions AI-generated and human-written text" to train the text classifier. For example, the classifier will label text as "very likely," "unlikely," "unclear if it is," "possibly" or "likely" AI-generated. 

As the OpenAI blog estimated only 26% of AI-written text was correctly identified. It also flagged 9% of human-written text as being composed by AI. 

That said, OpenAI's classifier has some limitations. For example, the Microsoft-backed company noted that the tool isn’t always accurate — AI-generated text can be edited to evade detection tools, and the text classifier may misidentify both AI-generated and human-written samples. Besides, it requires a minimum of 1,000 characters, which is approximately 150 - 250 words.  

The company also said it would sometimes “incorrectly but confidently” label human-written text as being from an AI, especially if it’s very different from anything in the training data. It makes it clear that the classifier is still very much a “work-in-progress".

OpenAI further acknowledged that the tool was trained using English text samples written by adults, so it may misidentify content written by children or in languages other than English. 

Our intended use for the AI Text Classifier is to foster conversation about the distinction between human-written and AI-generated content," the blog post said. "The results may help, but should not be the sole piece of evidence, when deciding whether a document was generated with AI." 

"Because of these limitations, we recommend that the classifier be used only as one factor out of many when used as a part of an investigation determining a piece of content’s source," it added. 

To be sure, OpenAI is not the first to come up with a tool for detecting ChatGPT-generated text. Edward Tian, a 22-year-old student at Princeton University, created a website called GPTZero in December 2022 that can detect if a piece of writing has been created using the AI tool ChatGPT, in other words, AI plagiarism.  

Also, plagiarism detection tool Copyleaks launched its own AI Content Detector this month for educational institutions and publishing. The Giant Learning Model Test Room, a 2019 collaboration between the MIT-IBM Watson AI Lab and the Harvard Natural Language Processing Group, identifies AI-generated writing using predictive text. 

Nonetheless, one area OpenAI is really focusing on with this detection tech is education. The blog said that “identifying AI-written text has been an important point of discussion among educators,” as different schools have reacted to ChatGPT by banning or embracing it. 

The company said it is already “engaging with educators in the US” to figure out what they see from ChatGPT in their classrooms and is soliciting feedback from anyone involved in education. The tool will be available as a web app, along with some resources for teachers, the blog post said.