- AI writing tools like ChatGPT and Claude are widely used for emails, posts, and articles
- No single word or habit conclusively proves a text is AI-generated due to varied styles
- AI text often uses more scientific terms and longer sentences than human writing
Artificial intelligence-generated writing is becoming harder to identify as chatbots continue to change the way they produce text.
Tools such as ChatGPT, Claude, Gemini and Grok are now being used to write emails, social media posts, essays, news stories and even research papers. One estimate cited by The Economist said AI was being used to create more than a third of new websites.
The study found that there is no single word or writing habit that can prove a text was written by AI. Different chatbots have different writing styles, and those styles can change after software updates.
To find common patterns, The Economist asked ChatGPT, Claude, Gemini and Grok to rewrite some of its articles. The chatbots were not allowed to search the internet. They were given AI-made summaries of the articles as prompts.
Researchers then compared the AI-written text with human writing. The study looked at 55,940 sentences and about 1.2 million words.
They also compared AI writing with articles from CNN, The New York Times and The Washington Post. Excerpts from popular novels published between 1950 and 2022 were also included.
The researchers found that AI writing can often be identified by looking at the words, punctuation, sentence length and paragraph structure.
For example, Older chatbots were often known for using words such as "delve" and "tapestry". Newer models use these words less often. Instead, they are more likely to use words such as "significant", "increasingly" and "consequences". They also use less common words such as "interdependence" and "reindustrialisation".
AI-written text also uses more scientific and technical words, including "parameter" and "methodology". The study also found that AI often turns simple verbs into longer nouns. For example, instead of saying "expand", it may use "expansion".
Many believe that using too many em-dashes is a sign of AI writing. But the study found that this is no longer a good way to spot AI text. After recent updates, only Claude used em-dashes more often than human writers. ChatGPT used them less than the other writers in the study.
Researchers also found that AI writing often uses less punctuation. Chatbots use fewer commas and semicolons and rarely use brackets. They also tend to write longer sentences. The word "and" is used very often in AI-generated text. Chatbots also tend to use fewer direct quotes from experts.
In addition to this, AI-written sentences are usually longer. Chatbots also often use the same sentence patterns to make their writing sound interesting. One example is the "not X but Y" structure. They also often use "not only... but also" and groups of three points. ChatGPT and Claude used these patterns more often than the other AI models and human writers studied.
There are also tools that claim to identify AI-written text.
Pangram, one such company, claims its system is 99.98% accurate. It has also partnered with blogging platform Substack. But AI detectors can sometimes get it wrong. They may mark human writing as AI-generated and usually do not explain exactly why they reached their conclusion.
"With every update, our study shows, AI writing is becoming more similar to human prose," the report said. The study added that the best way to spot AI writing is to look for several signs together rather than focus on one word or punctuation mark.
