Watermarks are often presented as a practical way to identify AI-generated text. New research described by MIT Technology Review shows why that confidence may be premature: current watermarking methods for text can be tampered with in ways that make them far less dependable.
The issue is not that watermarking has no value. The issue is that attackers may be able to both erase a watermark from AI-written material and imitate one on text that should not be trusted as authentic.
What AI text watermarks are supposed to do
Watermarking for AI-generated text is designed to add hidden signals to writing produced by an AI system. These signals are not meant for human readers. They are meant for computers that can later inspect the text and judge whether it likely came from an AI model.
The source article describes watermarking as a fairly new invention that has already become a popular answer to concerns about AI-generated misinformation and plagiarism. The appeal is clear: if AI text can be marked at the moment it is created, then platforms, schools, publishers or regulators could have a technical clue about where that text came from.
The European Union’s AI Act, which enters into force in May, will require developers to watermark AI-generated content. That makes the reliability of these systems more than a technical question. It is becoming a policy and compliance question too.
But Robin Staab, a PhD student at ETH Zürich and part of the team that developed the attacks, says the new findings show that the cutting edge of watermarking technology does not yet meet what regulators need. The research is yet to be peer reviewed and will be presented at the International Conference on Learning Representations conference in May.
How the hidden pattern works
AI language models generate text by predicting the next likely word in a sentence. They do this one word at a time, building a response from those predictions.
The watermarking methods examined in the research work by changing how the model chooses words. The algorithm divides the model’s vocabulary into a “green list” and a “red list.” The AI model is then pushed toward words from the green list.
That creates a statistical pattern. If a sentence contains more green-list words, a detector may conclude that it is more likely to have been generated by a computer. Human writing, by contrast, tends to include a more random mix of words.
This approach is subtle because the marker is embedded in ordinary language. The text does not need a visible label. The signal is in the distribution of word choices.
What the researchers were able to break
The ETH Zürich team tested five different watermarks that use this green-list and red-list approach. According to Staab, the researchers were able to reverse-engineer the watermarks by using an API to access an AI model with the watermark applied and prompting it many times.
Those repeated responses gave the attacker enough information to build an approximate model of the watermarking rules. The process involved analyzing AI outputs and comparing them with normal text.
Once the researchers had an approximate view of which words might be watermarked, they carried out two types of attacks:
- Spoofing attack: They used the stolen watermark information to produce text that could be passed off as watermarked.
- Removal attack: They scrubbed AI-generated text of its watermark so it could be passed off as human-written.
The results were significant. The team had a roughly 80% success rate in spoofing watermarks, and an 85% success rate in stripping AI-generated text of its watermark.
In practical terms, that means a watermark could fail in two directions. It could make untrustworthy text look more legitimate, or it could let AI-generated text avoid detection.
Why this matters for trust
The central promise of AI text watermarking is trust. A detector is useful only if people can rely on its signal. If the signal can be copied or removed, then it may give users confidence in text they should question.
The source article notes that researchers outside the ETH Zürich team have also found watermarks to be unreliable and vulnerable to spoofing attacks. One of them is Soheil Feizi, an associate professor and director of the Reliable AI Lab at the University of Maryland.
Feizi says the ETH Zürich findings confirm that these problems continue and also apply to the most advanced types of chatbots and large language models being used today. The research “underscores the importance of exercising caution when deploying such detection mechanisms on a large scale," he says.
That caution is important because watermarking is being discussed as a response to misinformation and plagiarism. A weak detector could create a false sense of security. It could also create disputes over whether a text was written by a person or a machine.
Watermarks still have a role
The findings do not mean watermarking should be abandoned. Nikola Jovanović, a PhD student at ETH Zürich who worked on the research, says watermarks remain the most promising way to detect AI-generated content.
But the research makes the limits clear. More work is needed before watermarking is ready for large-scale deployment. Until then, expectations should be realistic.
Jovanović’s view is pragmatic: “If it’s better than nothing, it is still useful,” he says.
That may be the most useful way to understand AI text watermarks right now. They can help, but they are not a complete answer. If policymakers, developers and institutions treat them as a final solution, they may lean too heavily on a tool that attackers can already manipulate.