An Authors Guild test of AI detectors highlights a central problem for writers and publishers: tools built to identify machine-written text can produce very different answers on the same human work. Some systems in the test avoided false accusations entirely, while others treated every article as mostly AI-generated.
What the Authors Guild tested
The test used ten Guild articles published between 2020 and 2022, before generative AI went main stream. That timing matters because the articles were selected from a period before AI-assisted writing became a common concern in publishing workflows.
According to the source article, Pangram and Grammarly correctly identified every human-written text as human. Originality.ai also performed well in the test.
The weaker results were stark. Sidekicker delivered the worst performance: every article was flagged as mostly AI-generated, and two scored 100 percent. ZeroGPT was also described as unreliable because it sometimes reported high AI percentages across all the human-written texts.
Those results do not settle whether any detector can reliably catch AI-generated writing. They show something narrower but important: when the input was known human writing, some tools handled the task cleanly, while others created damaging false positives.
Why false positives matter
The Authors Guild warns that even the strongest-performing tools should not be used as the only basis for a consequential decision. AI detectors change over time, and their reliability cannot simply be assumed.
For authors, the risk is not abstract. A false accusation can affect contracts and reputations. If a publisher, platform, school, or client treats a detector score as proof, a writer may be forced to defend work that was never machine-generated in the first place.
The Guild’s concern is also procedural. Publishers should disclose how they evaluate suspected AI use and give authors a meaningful opportunity to respond. That kind of process matters because the detector’s output is not the same thing as evidence that can explain itself.
Pangram CEO Max Spero recently described his detector as essentially a black box. In other words, even when a tool produces a confident result, it may not be able to explain in detail why a passage was flagged as AI-generated.
The professional-writing paradox
The source article points to a difficult overlap between polished human writing and AI output. Professional writing often follows statistical patterns that language models have learned, because those models were trained on that kind of writing.
That creates a paradox for skilled authors. A writer who has spent years improving clarity, precision, structure, and economy may produce work that resembles the type of writing AI systems are designed to imitate.
Max Spero has said language models can reveal themselves through uniformity, especially in the way they build arguments, while humans write with more variety. But the Authors Guild’s warning suggests the boundary is not always safe enough for high-stakes decisions.
The issue is not only whether a detector is right in many cases. The practical question is what happens when it is wrong. A single false positive can carry more consequences for an individual writer than a general accuracy claim can repair.
What the results do and do not prove
The strong results from Pangram, Grammarly, and Originality.ai should be read carefully. The test shows that these tools were able to recognize the selected human-written texts as human. It does not prove that they are equally effective at catching AI-generated text.
A detector may be tuned to reduce false positives. That can protect human authors from being wrongly flagged, but it may also allow some AI-written or AI-assisted work to pass through. The source article makes clear that many texts written by or with AI could still go undetected.
That distinction is essential for publishers and editors. A detector that rarely accuses human writers may still miss machine-generated content. A detector that aggressively flags suspicious patterns may create more false accusations. The Authors Guild test shows how wide that tradeoff can be across different products.
- Pangram and Grammarly identified all ten human-written articles as human.
- Originality.ai also performed well on the tested human texts.
- Sidekicker marked every article as mostly AI-generated.
- ZeroGPT produced unreliable results with sometimes high AI percentages.
The wider debate over AI writing
The usefulness of AI detectors remains contested because errors are expected to continue. The debate is also complicated by the fact that AI can be a useful writing tool, while public discussion often blends together using AI to write and using AI to think.
Detector advocates, including Max Spero, argue from the idea of a social contract between writer and reader. In that view, the writer invests time and effort to shape an idea, and the reader invests attention in return. If AI reduces the cost of producing text to zero, the incentive to flood the internet with low-value content becomes stronger.
But the source article raises a separate question: whether the value of writing comes mainly from typing, or from the topic, idea, perspective, story, research, argument, and judgment behind the finished work. That question is not answered by a detector score.
The Authors Guild test therefore points to a practical middle ground. AI detectors may have a role, but their results need context, disclosure, and human review. For writers, the key issue is not whether detection exists. It is whether institutions treat detection as a clue or as a verdict.