Generative AI has made it easier to produce large volumes of misleading material, adding pressure to efforts to identify and limit disinformation online. At a TechCrunch Disrupt 2023 panel, NewsGuard’s Sarah Brandt and Adobe’s Andy Parsons discussed the risks and possible ways to respond.
More content, lower barriers
Online disinformation existed before generative AI. Parsons pointed to a 2019 viral clip of former House Speaker Nancy Pelosi (D-CA), altered to make her appear to speak in a slurred, awkward way. The newer concern, Brandt said, is that AI tools can make creating and distributing misleading content cheaper and simpler at scale.
NewsGuard rates news and information websites and provides services including misinformation tracking and brand safety for advertisers. In May, it identified 49 news and information sites that appeared to be almost entirely written by AI tools. Since then, it has found hundreds of additional unreliable, AI-generated websites.
Some of these sites publish at a pace that makes individual claims harder to track. Parsons described operations producing hundreds, and in some cases thousands, of articles a day. Some seek search visibility and programmatic ad revenue; others spread misinformation or disinformation.
That means the challenge is not only whether a single fabricated story can persuade someone. A steady stream of content can also make it more difficult to distinguish dependable reporting from unreliable material, especially when misleading stories are produced in formats that look familiar.
Convincing false narratives
NewsGuard also tested how text-generating models responded to prompts. A study published in late March found that OpenAI’s flagship model GPT-4 was more likely than GPT-3.5 to spread misinformation when prompted. The test found GPT-4 could present false narratives more convincingly across formats including news articles, Twitter threads and TV scripts.
The formats in the test mimicked Russian and Chinese state-run media outlets, health hoax peddlers and well-known conspiracy theorists. The finding points to a practical difficulty: misleading material does not have to look like an obvious hoax. It can be shaped to resemble the kinds of content people already encounter.
These results do not mean every AI-generated item is false, or that every model will respond the same way in every situation. They do show why the panelists treated the falling cost and increased volume of production as central concerns alongside the content of any one post.
Safeguards and signs of origin
Parsons said Adobe uses safeguards such as filters in its generative AI product family, Firefly, to try to prevent misuse. Adobe also co-founded the Content Authenticity Initiative (CAI) in 2019 with the New York Times and Twitter. The initiative promotes an industry standard for provenance metadata, information that can help show where content came from.
Adoption of the CAI standard is voluntary. Safeguards used by one company do not ensure that other developers will adopt similar measures, and those protections may be bypassed. That leaves open questions about how broadly provenance information will be included and how useful it will be when content travels across platforms.
Watermarking is another approach discussed by the panelists. DeepMind proposed SynthID, a standard for marking AI-generated images in a way that is imperceptible to people but detectable with a specialized tool. French startup Imatag offers a watermarking tool it says is resilient to resizing, cropping, editing and compression. Steg.AI uses an AI model to apply watermarks designed to survive resizing and other edits.
Watermarks can offer a signal about an image’s origin, but the panelists did not present them as a complete solution. They are one part of a broader effort to make AI-generated media easier to identify and to encourage more responsible deployment.
Trust is an incentive, not a guarantee
Brandt was optimistic that economic incentives could encourage AI companies to build and deploy their tools more thoughtfully. If users find that a system hallucinates, spreads misinformation or fails to cite sources, she argued, they may see its output as less reliable than content from a company making efforts to improve trustworthiness.
That argument puts user trust at the center of the business case for safeguards. But incentives may vary across developers and uses, and voluntary standards depend on participation. The panel discussion left the answer open, particularly as safeguard-free open source generative AI models become widely available.
For now, the measures described—filters, provenance metadata and watermarks—offer ways to reduce or identify some misuse. The scale of AI-generated publishing and the uneven adoption of safeguards mean the work of judging online information still matters.