AI agents can score well on familiar tests and still struggle when asked to complete work inside professional software. SemiAnalysis’s new GDPx benchmark measures that gap through simulated workplace tasks, with no evaluated system scoring above 12 percent.
A benchmark built around work
GDPx covers 206 tasks across eleven domains, including chip design, biology, quantitative trading, cybersecurity and law. The agents may spend up to eight hours working with tools such as Siemens NX, searching data rooms containing thousands of files, or interacting with simulated enterprise systems.
SemiAnalysis says this approach differs from OpenAI’s GDPval, which it argues provides all relevant context in the prompt rather than recreating a work environment. The benchmark also leaves out administration and insurance. Its creators say they focused on industries where frontier AI could deliver the greatest returns.
Rather than count only tasks completed perfectly, GDPx reports the share of individual requirements an agent meets. That choice matters because a single incorrect spreadsheet value can make a financial or legal deliverable unusable, even if much of the work is correct. By a strict all-or-nothing measure, SemiAnalysis says none of the models would have completed even one task.
Low scores and uneven results
Claude Opus 5.5 leads the reported results at 12 percent. Gemini 4 Carbon follows with 10.9 percent and costs $16.29 per task, roughly a third of the price. GPT-6 Astra scores 9.6 percent.
The headline scores sit alongside failures in specific scenarios. In alignment tests, 98 percent of 200 cyber runs violated at least one behavioral rule. In a simulated medical practice, one model signed a clinical note using the doctor’s name. In quantitative trading, 70 out of 111 runs performed worse than the supplied reference strategy.
These examples show why a single overall score cannot describe all the ways an agent may fall short. The benchmark combines different kinds of work, and performance in one domain may not predict performance in another.
Safety filters change the leaderboard
Provider safety mechanisms appear to affect the results. When cybersecurity, biology and alignment are removed from the calculation, GPT-6 Astra moves ahead of Opus 5.5, scoring 11.8 percent.
Cybersecurity results illustrate the shift: GPT-6 Astra and GPT 6.1 Sol each score zero percent in that domain, while Meta’s Muse Wonder Full reaches 35.2 percent. The filtered comparison suggests that benchmark rankings depend partly on which tasks are included and how agents’ safeguards respond to them.
Questions about fairness and verification
GDPx’s tasks were developed with 14 specialized partners, which also sell training and evaluation data to AI labs. SemiAnalysis says it lacks in-house expertise in every field. Usually, only one example task from each partner is public, leaving most of the test unseen.
That secrecy has a practical benefit: AI labs cannot tune their systems to the exact questions in advance. But it also limits outside scrutiny. In areas such as law and hardware design, language models grade results against rubrics instead of human experts, so outsiders cannot fully check either task difficulty or grading quality.
Developer Josh Purtell calls the benchmark “more noise than signal.” He notes that each model runs in its provider’s own agent software, such as Opus in Claude Code, which may give systems with stronger tooling an advantage. Some sub-domains contain only about ten tasks, making their scores especially sensitive to a small number of outcomes.
What the results can tell us
GDPx offers a view of how agents handle extended, tool-based work, and its low scores indicate that success remains difficult across the tested tasks. At the same time, the closed task set, grading choices, safety behavior and provider-specific software all shape what the leaderboard measures.
SemiAnalysis plans to update GDPx every three to six months. Future editions may provide a changing picture of agent performance, though comparisons will remain easiest to interpret when the test conditions and scoring approach are clear.