Large language models are becoming more capable at tasks that psychologists use to probe how people infer what others think, intend, or misunderstand. New research published in Nature Human Behavior found that some AI models performed as well as, and in some cases better than, humans on tests associated with theory of mind.
The result is striking, but it comes with an important limit. The study does not show that AI systems actually understand human feelings or possess a human-like mind. It shows that some models can produce strong answers on structured tasks built to measure social and mental-state reasoning in people.
What theory of mind tests are trying to measure
Theory of mind is linked to emotional and social intelligence. It helps people infer intentions, recognize misunderstandings, engage with others, and empathize. Most children develop these kinds of skills between three and five years of age.
Psychologists have created many tests to study this ability. These tests often ask whether someone can understand the difference between what is said and what is meant, recognize when a person holds a false belief, or notice when a social mistake has occurred.
The researchers in this study applied that same systematic approach to large language models. Their goal was not to prove that models have minds, but to examine how well they perform on the kinds of tasks used to assess theory of mind in humans.
How the models and humans were tested
The team tested two families of large language models: OpenAI’s GPT-3.5 and GPT-4, along with three versions of Meta’s Llama. They also tested 1,907 human participants so the model scores could be compared with human performance.
The study used five types of tasks:
- Hinting task: testing whether a person or model can infer real intentions from indirect comments.
- False-belief task: testing whether someone can infer that another person may believe something that is not true.
- Faux pas recognition: testing whether someone can identify a social mistake.
- Strange stories: testing whether someone can explain the gap between what was said and what was meant when a protagonist does something unusual.
- Irony comprehension: testing whether someone can understand irony.
The AI models received each test 15 times in separate chats. That setup was meant to make each request independent. Their responses were scored using the same method applied to human answers.
Where GPT and Llama performed differently
The results were mixed across model families and task types. Both versions of GPT performed at, or sometimes above, human averages on tasks involving indirect requests, misdirection, and false beliefs. GPT-4 outperformed humans on the irony, hinting, and strange stories tests.
Llama 2’s three models performed below the human average overall. But there was one notable exception: Llama 2, the biggest of the three Meta models tested, outperformed humans in recognizing faux pas scenarios.
GPT had the opposite pattern on faux pas recognition. It consistently gave incorrect responses. The authors believe this may reflect GPT’s general reluctance to generate conclusions about opinions, because the models often responded that there was not enough information to answer either way.
That split matters because it shows that AI theory of mind performance is not a single capability that rises evenly across every task. A model may be strong at recognizing indirect meaning while failing on a social judgment task. Another model may lag overall yet do well in one specific category.
Why strong scores do not equal human understanding
The research arrives as AI assistants are being designed to feel more natural in conversation. OpenAI and Google announced GPT-4o and Astra last week, with both systems intended to deliver smoother, more naturalistic responses than earlier assistants.
More natural responses can make AI systems seem useful and empathetic. But the source article stresses the risk of confusing performance with human-like mental life. Cristina Becchio, a professor of neuroscience at the University Medical Center Hamburg-Eppendorf who worked on the research, warned that people naturally tend to attribute mind, mental states, and intentionality to entities that do not have a mind.
That distinction is central. A model can generate an answer that looks socially aware without experiencing awareness. It can identify implied meaning in a test without understanding a person in the way another person might.
The study therefore points to a practical tension. Better performance on theory of mind tests may make AI assistants appear more responsive in everyday interactions. At the same time, users and developers need to avoid treating those outputs as proof that the system has human-like understanding.
The benchmark problem behind the result
Several researchers who did not work on the study urged caution about what the tests actually reveal. Maarten Sap, an assistant professor at Carnegie Mellon University, noted that these psychological tests are well established and may have appeared in the models’ training data. A child taking a false-belief test has probably never seen that exact task before, while a language model might have.
Tomer Ullman, a cognitive scientist at Harvard University, also emphasized that research like this can help clarify what large language models can and cannot do. But he argued that outperforming a human on a theory of mind test does not mean an AI has theory of mind.
The larger message is not that benchmarks are useless. It is that passing a benchmark can mean different things for humans and machines. For people, these tests are tied to development, social intelligence, and lived interaction. For large language models, strong scores may reflect patterns learned from language and training data rather than human-like reasoning.
That makes the finding important but not definitive. The models are getting better at tasks built around mental-state inference, indirect meaning, irony, and social judgment. What remains unresolved is how they arrive at those answers, and whether current tests can separate genuine human-like understanding from convincing imitation.