A benchmark called TasteVal is drawing attention to a capability that could shape how quickly AI improves: the ability to plan experiments and learn from their results. Its findings have led Anthropic co-founder Jack Clark to move his prediction for recursive self-improvement forward by a year, to 2027.
What TasteVal tests
P-Zero Research uses the phrase “research taste” for three skills: choosing worthwhile research questions, designing experiments to answer them, and interpreting the results correctly. TasteVal assesses only the latter two. The research goal is set in advance, so the model does not have to decide which scientific direction deserves attention.
The benchmark includes eight tasks drawn from AI model development. They cover areas such as selecting training data, pretraining and fine-tuning language models, training on human preferences, and defending against adversarial prompts. For each task, the model chooses which experiment to run and uses what it learns to guide its next step.
To limit the influence of coding ability, a fixed coding agent runs the experiments on one Nvidia H100 GPU. Each run ends after 40 GPU-hours of compute or 120 hours of wall-clock time.
The human comparison group consisted of 24 experts who had recently worked at places like OpenAI, Google DeepMind, Nvidia, and Microsoft Research. At least two people worked on each task, and the best human attempt for each task set the comparison score.
Efficiency is the benchmark’s measure
P-Zero Research measures research taste through compute efficiency. The idea is that when experiment compute is limited, a model that chooses useful tests and draws sound conclusions can make more progress with the same resources.
In this framework, the compute multiplier describes how much less compute a model needs to reach the experts’ scores. Anthropic’s Opus 5.5 achieved a multiplier of 2.3, reaching those scores with less than half the compute. The reported statistical range, however, runs from 1.15 to 4.37, meaning the estimated advantage is uncertain. A single model run cost roughly one-thirtieth of what an expert attempt cost on average.
Across several model generations, the measured multiplier roughly doubled every three months since December 2025. The estimated doubling time ranges from 1.7 to 5.0 months. The trend was much slower in prior years, with a steeper curve appearing around the turn of 2025 into 2026.
A narrow result with a large caveat
The benchmark does not show that AI systems are already improving themselves. It covers a limited slice of scientific work, and it gives models the research question rather than asking them to identify the most valuable question to pursue.
That distinction matters. Choosing a promising direction is often seen as central to scientific intuition, but TasteVal leaves that decision to the benchmark. Its tasks can also be checked quickly and cheaply, which the authors say may make them easier for AI labs to optimize. They acknowledge that the results may overstate how quickly research taste is improving, and the comparison group did not include top-tier researchers.
Other evidence in the article also points to a gap between proposing research methods and making final choices. A study led by a team at Fudan University found that AI agents contributed up to 55 percent of methodology proposals in the development of the researchers’ own model. Humans still made over 85 percent of final decisions. The study’s authors said it remained unclear how AI could generate diverse research directions and judge their value before results were available.
Why Clark changed his forecast
Clark, who co-founded Anthropic and writes the Import AI newsletter, has argued that creativity is central to how quickly AI systems might improve themselves. He now points to progress in mathematical breakthroughs and benchmarks such as TasteVal as signs that this capability may be changing.
In a long essay in May, Clark put the odds of a system independently training a stronger successor at roughly 60 percent by the end of 2028 and 30 percent by 2027. He now believes recursive self-improvement could become possible as early as 2027, earlier than his previous expectation of 2028. His connection to Anthropic, whose Opus 5.5 tops the benchmark, is a reason to consider potential bias in that forecast.
The result speaks to a dispute over whether AI can take on more than junior research tasks. TasteVal offers evidence that models can plan and evaluate experiments under set conditions. It does not establish that they can choose long-term research agendas or consistently judge which big questions are worth pursuing.
If the measured trend holds up in other tests, AI research could become faster and cheaper. But as models take on more of the work of planning and evaluating experiments, it could also become harder for people to verify how research decisions were made. TasteVal shifts attention from performance on defined tasks toward the question of how much independent research work models can handle.