AI text detectors can lose accuracy when writing changes or when AI-generated text is altered to look human. Researchers from MIT and Cornell University tested a way to help detectors adapt to new examples after deployment, without retraining them from scratch. In their experiments, the method performed better than traditional approaches, especially on text designed to evade detection.
Why detectors can fall behind
AI text detectors are themselves AI models. Developers train them on examples labeled as human-written or AI-generated, teaching them to distinguish between the two. But once a detector is released, the writing it encounters may differ from its training material.
That difference is known as a distribution shift. Human writing habits can change over time, and new large language models can produce different kinds of text. A detector trained on older examples may then misread newer writing, including human writing, because it does not match familiar patterns.
Detectors also face deliberate attempts to fool them. People can use automated “humanizer” tools to revise AI-generated writing so it appears more like human writing to a detector. Developers may respond by adding examples of this disguised text to training data, setting off a cycle of adaptation and evasion.
Learning from new, unlabeled text
The team studied a technique called test-time adaptation, or TTA. Rather than keeping a deployed detector fixed, TTA lets it adjust to new data as it arrives. For a text detector, that data consists of new pieces of writing submitted by users.
The incoming examples are unlabeled: developers do not know which were written by people and which by AI. The researchers’ approach uses these examples to help the detector adapt to current writing patterns without starting training from scratch.
To evaluate the method, the team created datasets containing both human and AI writing, including AI-generated examples designed to evade a detector. The model adapted using the unlabeled data, and the researchers then tested it against data with known labels.
Stronger results against disguised text
In experiments meant to represent gradual changes in human writing, a detector using TTA reached 85 percent accuracy. Traditional methods reached 73 percent accuracy in the same setting.
The gap was larger when the text was deliberately disguised. A commercial detector identified about 27 percent of the disguised AI-generated text in the experiment. With TTA, the researchers’ approach identified more than 93 percent.
Those results suggest that adapting to new examples could help detection systems keep pace with attempts to bypass them. The researchers say the method could be relevant to educators, editors, publishers, and hiring managers concerned about AI-generated writing presented as human-written. They also point to possible uses in cybersecurity, including efforts to prevent email phishing scams or identify search engine optimization fraud.
What the method cannot settle
The researchers caution that test-time adaptation may not always work. Future language models may produce text that sounds more natural, while verifiable human-written examples for training could become harder to find. Without reliable examples, adapting a detector to changing writing becomes more difficult.
Shifts in human writing may also be harder to handle than deliberate attempts to evade detection. The source describes those changes as slower and subtler, which can make them more challenging for TTA to address.
The research will be presented at the Conference on Neural Information Processing Systems. The team includes lead author Kevin Ren, a graduate student at Cornell Tech; study co-author Manish Raghavan of MIT; and senior author Nikhil Garg, an assistant professor at Cornell Tech.
The work offers a way to make deployed detectors more responsive, but it leaves a broader question open: how reliably can people identify who wrote a piece of text if human and AI writing become difficult to distinguish? The researchers plan to explore other strategic domains for TTA and develop better benchmarks for evaluating detection models as conditions change.