False tweets generated by AI were slightly harder for people to identify than false tweets written by humans in a study of 697 participants. The finding points to a possible advantage for disinformation creators, while leaving open how large that advantage is and what causes it.
What the study asked people to judge
Researchers chose common disinformation topics, including climate change and covid. They asked OpenAI’s large language model GPT-3 to produce 10 true tweets and 10 false ones, then gathered a random sample of true and false tweets from Twitter.
Participants took an online quiz. They had to judge whether each tweet came from AI or Twitter, and whether its content was accurate or contained disinformation. The researchers found that participants were 3% less likely to believe human-written false tweets than AI-written ones.
The difference is small, but it suggests that the source and construction of a message may affect how readily people accept it. The study does not explain why the AI-generated false tweets were more likely to be believed.
Structure may make a message easier to process
Giovanni Spitale, the University of Zurich researcher who led the study, said the way GPT-3 organizes information could be one explanation. He described its text as more structured than human-written text and also more condensed, which may make it easier to process.
That is a possible explanation, rather than a settled account of the result. The researchers remain unsure why participants responded differently to AI-written and human-written false tweets. The study raises a question about how writing patterns shape credibility, but it does not establish a single mechanism.
Spitale said he believes the difference might be larger if the team repeated the research with OpenAI’s latest large language model, GPT-4, given its greater power. That is his expectation; the reported study tested GPT-3.
More output could make the problem harder to manage
Generative AI tools make it possible to produce text quickly and cheaply. That accessibility could help bad actors create false narratives for conspiracy theorists or disinformation campaigns. If AI-generated messages are also somewhat more convincing, the combination could make online disinformation easier to produce and distribute.
Detection remains an uncertain part of the response. AI text-detection tools are still in early development, and many are not entirely accurate. OpenAI has acknowledged the risk of its tools being used for large-scale disinformation campaigns. The company’s report, released in January, warned that it is “all but impossible to ensure that large language models are never used to generate disinformation.”
At the same time, the size of the risk is not yet clear. The authors of OpenAI’s report called for further research into which populations may be most at risk from AI-generated inauthentic content, and how model size relates to the performance or persuasiveness of its output.
Persuasive writing does not guarantee broad influence
Jon Roozenbeek, a misinformation researcher at the University of Cambridge who was not involved in the study, cautioned against panicking. AI could make online disinformation easier and cheaper to create than human-staffed troll farms, but platform moderation and automated detection systems can still obstruct its spread.
A tweet that is slightly more persuasive does not automatically mean that people are ready to be manipulated at scale. The study measured participants’ judgments in an online quiz; it does not, by itself, establish how AI-generated disinformation would affect people across populations or campaigns.
The result is a reason to keep investigating how people assess AI-written messages. It is not a complete measure of the threat. Understanding who is most susceptible, how text generation affects belief, and whether newer models change the pattern will require further research.