World models are often treated as a core ingredient for autonomous AI agents: systems that can anticipate what will happen after an action. A new paper argues that many current approaches leave out something essential for predicting human behavior: the mental states of the people in the scene.
The claim is not that AI should simulate consciousness. The point is narrower and more practical. If an agent cannot represent what someone believes, wants, notices or considers appropriate, it may describe a scene correctly while still guessing the wrong next action.
Why physical prediction is not enough
According to the paper, existing world models such as Sora, Genie 3, JEPA and Marble focus on the physical layer of the world. They model objects, positions, motion and occlusion, but they do not include beliefs, goals or social expectations in their state space.
That omission matters most when AI systems are expected to work around people. The source highlights service robots, medical assistants and collaborative agents as examples where hidden mental states can shape what humans do next.
The paper uses a simple cup scenario to show the problem. If a cup is moved into a cabinet while someone is not looking, a purely physical model can still track the cup correctly. But it may fail to predict that the person will act based on an outdated belief about where the cup is.
In that kind of scene, the next state is not only physical. It is also mental. The visible world has changed, but the person’s belief has not changed in the same way.
What Mental World Modeling adds
The proposed framework is called "Mental World Modeling" (MWM) and is published on GitHub. It expands the world model concept by adding mental variables, including beliefs, attention, goals, intentions, emotions, norms and social relationships.
The framework separates what an agent can see from the fuller state maintained by the model. The target agent receives an egocentric partial view, while the world model tracks a more complete picture of the scene.
MWM also treats actions as having two layers. One is the physical carrier, such as speaking, pointing or grasping. The other is the mental payload, such as comforting, deceiving or rejecting.
This distinction helps explain why the same visible movement can mean different things. Sliding a cup across a table could be an apology, a deception or an act of care, depending on the surrounding mental and social variables.
The authors are careful about the status of those variables. Mental states are not direct measurements. They are hypotheses inferred from behavior and context, so systems using the framework should preserve uncertainty and make their assumptions transparent.
How MENTIS tests the idea
To evaluate the approach, the researchers built MENTIS, a modular pipeline that requires no additional training. It divides action prediction into six steps, beginning with scene parsing and rendering the ego perspective.
The pipeline then separates action options into physical and mental components. It simulates the resulting states in parallel and scores each branch on three criteria:
- physical plausibility
- mental consistency
- social appropriateness
After those scores are produced, the system makes a deterministic decision. Each stage writes a machine-readable intermediate result, which makes it possible to locate errors in a specific part of the process.
The evaluation dataset, Menti-Bench, contains 448 decision scenes. These include 320 text descriptions, 100 picture stories and 28 sound-video clips. Each scene has six response options and a human-created reference solution that records both the correct action and the mental and physical states behind it.
The dataset is built around situations where mental tracking matters. 78 percent of the scenes involve at least two characters, which means many examples require some form of social or interpersonal reasoning.
The results favor explicit mental modeling
The researchers tested eight language models, including five from OpenAI and three from Anthropic. The OpenAI group included GPT-5.6-Sol and GPT-4.1, while the Anthropic group included Claude Fable 5, Claude Opus 4.8 and Claude Haiku 4.5.
The evaluation used F1 score, combining precision and recall for the selected action. Direct answers scored 63.3 across all models. Self-consistency, where the model answers the same question six times and selects the most common answer, raised the score to 77.9. The full MWM pipeline reached 87.9, while humans scored 98.5 under the same protocol.
The source emphasizes that the improvement is not simply a result of more retries. The weakest model using MWM, GPT-4.1 at 84.9, outperformed the strongest model using direct answers with self-consistency, GPT-5.6-Sol at 83.6.
Additional tests supported the importance of both channels. Removing the mental channel caused an average drop of 12.1 points. Removing the physical channel caused a 16.5 point drop. Predicting both transitions independently instead of coupling them cost 6.4 points.
The gains were largest where the theory would expect them to be largest. In interpersonal scenes, the F1 score improved by 26.4 points. In object-focused scenes, the gain was 14.0. Weaker base models benefited more from the explicit structure: the gap between MWM and direct answers was 28 points for GPT-4.1 and 21 for GPT-5.6-Sol.
Where the gap remains
The framework still does not fully match human performance. To find out why, the authors replaced individual pipeline stages with the human reference solution.
The biggest gain came from perfect state transitions, at +3.5 points. Perfect initial state added +2.8, and perfect observation added +1.7. When all intermediate steps were replaced with the reference solution, the pipeline reached 97 points.
About 80 percent of the remaining gap was linked to prediction errors in intermediate stages, especially transition simulation. In other words, the hard part is not only describing the starting scene. It is predicting how the physical and mental layers change together.
The paper arrives as world models remain a contested area. Demis Hassabis has said he spends most of his research time on the topic and expects a "ChatGPT moment" for these systems. At the same time, the field has not settled on a single definition of what counts as a world model.
The new work places Sora, Genie and JEPA in the same broad family and argues that they share a key blind spot. For AI agents that need to act around people, modeling the world may require modeling not just where things are, but what people think is true.