GPT-4 has crossed a familiar threshold in a new version of the Turing test: after short chats, human participants could not reliably tell whether they were speaking with the AI or with another person.
The finding is not only about GPT-4. It also shows how easily people can misread human conversation when they are actively looking for signs of artificial intelligence.
A test built around five-minute conversations
The study used a randomized, controlled two-player variant of the Turing test, 74 years after Alan Turing proposed his famous thought experiment comparing humans and machines.
Participants had five-minute conversations and then judged whether the other side was human or artificial. The language models were prepared with a detailed prompt before the test. That prompt was designed to shape their answers, including by limiting typical model behavior and making them occasionally produce spelling mistakes.
Those details matter because the test was not simply measuring a raw system in isolation. It was measuring how a model behaves in conversation when prompted to appear less obviously machine-like, and how humans respond when the task is to detect that difference.
GPT-4 crossed the study’s success line
The researchers defined 50 percent as success on the Turing test. At that level, participants are no better than chance at distinguishing human from machine.
By that definition, GPT-4 passed. It was judged to be human 54 percent of the time. GPT-3.5 reached 50 percent, while ELIZA, the simple rule-based reference system from the 1960s, reached 22 percent.
Human participants still performed better than the systems, but not perfectly. They were judged human 67 percent of the time, which means they were rated as AI in just under a third of cases.
The study therefore points in two directions at once:
- Modern language models can appear human enough to mislead participants in short chats.
- People can become so cautious about AI that they wrongly reject real humans as artificial.
That second point is one of the most striking parts of the result. When people know that AI can behave in a human-like way, they may raise their standard for what counts as human. Ordinary human phrasing, hesitation, or style can then become suspicious.
Confidence did not solve the problem
The participants were not merely making random guesses. When they identified GPT-4 as human, they did so with an average confidence of 73 percent.
That detail makes the result more important. A mistaken judgment held with confidence can be harder to correct than an uncertain guess. In the setting described by the study, people were explicitly focused on detection, yet they still frequently misidentified the source of the conversation.
The Turing test is often treated as a benchmark for machine intelligence. The study complicates that reading. According to the source article, the results support the view that the test may reveal participants’ strategies and reasoning more than it reveals intelligence itself.
Participants tended to focus more on language style and socio-emotional factors than on knowledge and logic. However, those who asked about logic, current events, or human emotions and experiences were more often correct on average.
That suggests a practical lesson inside the experiment: surface-level conversational style can be misleading. More targeted questions may help, but even then the test did not produce reliable separation between GPT-4 and humans.
Why deception is the bigger issue
The study’s implications go beyond whether GPT-4 passed a famous test. The deeper issue is whether language models can credibly imitate people in settings where users may not be carefully checking for deception.
"The results here likely set a lower bound on the potential for deception in more naturalistic contexts where, unlike the experimental setting, people may not be alert to the possibility of deception or exclusively focused on detecting it,"
That warning is central. In the study, participants knew detection was the point. In other settings, people may simply assume a conversation is authentic, especially if the interaction feels familiar, fluent, or emotionally responsive.
The source article notes that systems able to mimic humans could have broad economic and social effects. One example is customer contacts that had previously been reserved for human employees.
The risks also include misleading the public, misleading human operators, and weakening social trust in genuine human interactions. If people cannot easily tell when they are speaking with a person, they may become more suspicious of everyone. The study shows that this suspicion can already spill over onto real humans.
What the Turing test now tells us
The result does not settle whether language models are intelligent. The source article makes clear that the Turing test has long been criticized as too easy, too difficult, or not truly a measure of intelligence.
What the study does show is narrower and more concrete: in short conversations, GPT-4 could pass as human often enough to meet the researchers’ threshold, and people sometimes misclassified actual humans as AI.
That makes the Turing test less like a clean intelligence exam and more like a mirror for human judgment. It exposes the cues people trust, the cues they overvalue, and the uncertainty that appears when machine language becomes fluent enough to blend into ordinary conversation.
For anyone thinking about AI systems in public-facing roles, the lesson is direct. Human-like language is not a small design detail. It affects trust, detection, and the boundary people draw between authentic human interaction and machine-generated conversation.