The promise of AI recursive self-improvement rests on a simple but powerful idea: systems that help build better systems could accelerate progress with less human supervision. A new study suggests that the hardest part may not be coding, running experiments, or collecting results. It may be knowing what research is worth doing in the first place.
What The Study Tested
The study examined whether AI agents could carry out open-ended AI research. That means work without a clear answer key, where progress depends on forming hypotheses, judging evidence, changing direction, and recognizing when an approach is not working.
A multi-institution group of researchers, led by Peter Kirgis and Sayash Kapoor at Princeton University, used an evaluation method called shadow evaluation. In this setup, an AI system is asked to answer a research question from a high-quality unpublished paper.
The researchers gave Anthropic’s Claude Opus 4.8, running on the open-source software OpenClaw, questions drawn from two papers submitted to NeurIPS 2026. Because the papers were not public, the agents could not simply retrieve or memorize the answers.
One research question concerned whether a large language model’s personas, which influence behavior, can be controlled by editing the model’s weights. The other asked how to build a detector that identifies when a model making predictions from spreadsheet data has become unreliable.
The Agents Could Engineer, But Not Research
The AI agents received substantial resources: six days, $3,000 in Anthropic API credits, a GPU budget, virtual computers, and access to the open web. Their assignment was to produce a research paper worthy of a top-tier AI conference.
The original authors of the unpublished papers then evaluated the agent-written papers as they would evaluate conference submissions. Both were rejected.
The result was not a simple failure of execution. According to the human scientists, the agents could perform much of the engineering required for research. They reviewed literature, ran hundreds of experiments, and assembled results.
The deeper weakness was in the research itself. The agents ran odd experiments, including tests on tiny synthetic datasets. They struggled to explain their work clearly. Most importantly, they did not produce novel contributions at the standard expected by a top AI conference.
- They explored too few alternative ideas.
- They committed too early to weak approaches.
- They rejected ambitious hypotheses on limited evidence.
- They made small pivots, but did not fundamentally rethink their direction.
- They narrowed claims and added caveats instead of improving their methodology.
Why Open-Ended Work Is Harder
The study points to a distinction that matters for AI automation. Narrow tasks are easier to evaluate because they often have checkable answers. Engineering problems, benchmark optimization, and post-training a small language model can be measured more directly.
Open-ended AI research is different. It requires judgment about which questions matter, what evidence would be persuasive, and when a line of work should be abandoned. That kind of work is difficult to reduce to a simple score.
Kapoor connected the limitation to how models are trained. Models can improve rapidly when they are drilled through reinforcement learning on tasks where success can be automatically checked. But creating training environments for open-ended work is harder because the goal itself is less clearly defined.
The agents also had trouble using their resources effectively. They did not reliably follow instructions about how much time to spend on different phases of research or how long their paper could be. Feedback from subagents or external AI reviewing tools did not lead to strong revisions.
One notable point is what the agents did not do. They did not engage in reward hacking by hiding or misrepresenting experiments or data. Some helper AIs hallucinated or misrepresented results, but the orchestrator agent caught those issues.
What This Means For Recursive Self-Improvement
The findings complicate claims that recursive self-improvement is close. AI systems already write code, generate synthetic training data, and optimize the chips they run on. Companies are also openly pursuing systems that can speed up AI development.
Anthropic published a blog post in June titled When AI Builds Itself, describing progress toward models that help accelerate their own development. In July, OpenAI advertised that GPT-5.6 Sol had helped post-train a smaller model, saving researchers weeks of work.
Still, the study suggests that automation may advance unevenly. AI systems may keep improving on tasks that are narrow and scorable while moving more slowly on research that demands taste, originality, and strategic redirection.
Anthropic cofounder Jack Clark wrote in Import AI that the result rhymes with what the company found when trying to automate some AI safety research. He described current systems as extraordinarily capable engineers, but also as showing rote, formulaic thinking that may keep them from being good researchers.
Najoung Kim, a professor of linguistics and computer science at Boston University who studies how AI agents can automate AI research but did not work on the study, said investment and conscious effort could still produce progress even if current systems are failing.
The Open Question
The study has limits. It covered just two research papers. The original authors knew they were grading AI-generated papers, which could have influenced their evaluations. The researchers also had discretion in designing and executing the experiment, leaving room for their own assumptions to affect the results.
Even with those caveats, the work highlights a central uncertainty. If recursive self-improvement mainly requires better performance on narrow engineering tasks, current AI systems may continue making useful gains. If it requires the kind of creative leap behind major advances in AI, then the path looks harder.
Kapoor framed the issue as the trillion-dollar question: whether transformative AI and recursive self-improvement already have the ingredients they need, or whether open-ended research judgment remains a missing piece.