A new benchmark suggests that leading AI agents can handle many specialized theoretical physics calculations at a level comparable to experts in the relevant fields. But solving a well-defined problem is only one part of research: deciding what to ask, and which results matter, remains a harder test.
A benchmark of specialist calculations
Thirty-five researchers from institutions including the International Centre for Theoretical Sciences (ICTS), Harvard University, KU Leuven, and Google DeepMind released TheoPhysEval, a collection of 50 unpublished theoretical physics problems with verifiable answers. The authors say a leading specialist could typically solve one in about a day or less without AI assistance. Researchers from neighboring fields might need much longer.
The problems cover eleven subfields, from fluid flows and biological processes to particle physics and cosmology. They are designed to test routine research calculations: applying established methods to a slightly changed question. Some contributors adjusted values in calculations from their own recent papers, making it impossible to simply reuse the earlier result.
That work can still require specialized knowledge and custom computation. One black hole problem, for example, required finding an earlier paper, adapting equations from a five-dimensional to a four-dimensional spacetime, and calculating the result with a custom program. Coordinator Suvrat Raju, a co-author of the earlier paper, needed about a day to solve it.
GPT-6-Astra led the tested agents
The researchers tested commercially available agents without giving them human hints. GPT-6-Astra, running in OpenAI's Codex coding environment, answered 49 of 50 problems when it had web access. That is 98 percent, at an average cost of under $3 per problem and just under $116 for the full benchmark.
Anthropic's Opus 5.5, running in Claude Code, scored 92 percent. Google's Gemini 3.8 Flash reached 68 percent, while the open-weight models DeepSeek-V4.1-Flash and GLM 5.3 scored 54 and 42 percent.
Each agent also took the test without internet access. In that setting, Astra and Opus each scored 92 percent. Since many questions depend on formulas from papers that the models were not given, the authors see the small difference as evidence that leading language models have absorbed much of the theoretical physics literature.
Some errors showed the limits of relying on retrieved material without checking whether it fits the task. During the web search run, Gemini and GLM copied formulas from the black hole paper without adapting them. Those formulas applied to five dimensions, while the problem required four.
Answer keys and ambiguous questions complicate scores
The benchmark's perfect score was not entirely straightforward to define. One problem asked agents to find an equation that produces a given curve. The expert solution used special mathematical forms that count as the required differential equation only in a broader sense. The authors acknowledge that the question might seem “unfair,” but say it reflects the challenge researchers face when inferring underlying physics from imprecise data. Every tested model missed it.
In several other cases, follow-up discussions with experts uncovered small errors in the reference answers. After those answer keys were corrected, Astra's responses were right every time. This matters when interpreting benchmark results: a score may reflect flaws in the grading reference as well as a model's performance.
The authors point to a recent study finding that some theoretical physics benchmarks are graded incorrectly and can underestimate model capabilities. That may help explain why agents did better on TheoPhysEval than on some other benchmarks, including CritPt, even though the authors consider their own problems harder on average. The problem set and test setup are available as a dataset on Hugging Face.
Calculating is not the same as doing research
The researchers describe the results as a major milestone: some current models may already exceed top human researchers on concrete calculations in specialized areas. They note that AI agents have also solved important open problems in mathematics. Yet theoretical physics has not seen a comparable breakthrough.
The distinction, in their account, is that verifiable calculations tend to come near the end of a research process. Before one can calculate, someone must build a theory and choose a productive question. If an approach proves too difficult, a good researcher may move to a simpler related problem that remains interesting. The authors say current agents are not good at making those choices.
Physicists also develop a shared, though not unanimous, sense of which problems are worth pursuing. The authors call this “taste.” Agents asked to devise new problems can get bogged down in details that people would find uninteresting. The researchers stress that these observations cannot be quantified and do not predict how AI will develop. They believe agents could conduct top-level research on their own in the near future, but calculation alone will not be enough.
Training and access may change
The benchmark also raises practical questions for education. Problems of this kind could once have served as first projects for early-career researchers. The authors say they are no longer suitable for that purpose unless the aim is specifically to practice working through the material.
AI could also affect who has the resources to compete. Theoretical physics has historically required much less money than experimental research, helping keep opportunities relatively equal. The benchmark itself was inexpensive to run, but the authors expect harder problems to demand sharply more resources. If that happens, a group's budget could increasingly shape what it can accomplish.
The researchers argue that theoretical physics should discuss access and equal opportunity before AI reshapes the field. They suggest that as machines become better at calculation, the human ability to identify worthwhile questions and see promising directions may matter even more.