Adding more AI agents can make a system finish sooner, but it does not guarantee a better result. Benchmark findings from Vals AI show that, in most tested comparisons, teams of agents cost considerably more than a single agent without a statistically significant quality gain.
Higher costs, mostly unchanged scores
Vals AI evaluated GPT-6 Sol and Claude Opus 5.5 on the Vibe Code Bench. Each model was tested alone and in a team, at medium and maximum reasoning effort. Across the four comparisons, team setups cost between 1.8x and 5.1x as much as solo agents.
Only one team comparison produced a statistically significant improvement: GPT-6 Sol at medium reasoning, where the team scored 7.3 points higher. At maximum reasoning effort, teams gave neither Sol nor Opus 5.5 a meaningful advantage. The findings indicate that extra agents may add little when a model is already using substantial compute.
That does not mean teams never help. The results are tied to a particular benchmark and set of models, and the medium-reasoning Sol result shows that a team can improve a score under some conditions. But across these tests, the added expense was much more consistent than a quality gain.
Speed and quality do not scale in the same way
Anthropic's tests with Opus 5.5 also found diminishing quality gains as more agents were added. Larger teams reached a given performance level faster, but increasing a team from ten to 100 agents produced only a small score increase after 24 hours. Separate ProgramBench tests showed speed improvements alongside higher token use.
Fable 5.1 had stronger quality gains above ten agents on a Lean theorem proving task, although it scored below Opus 5.5 across all tests. On a knowledge base task, its score dipped slightly when the team grew from 30 to 100 agents. These results underline that scaling depends on the task: adding agents can help in one setting and yield little or even lower scores in another.
In comments on the Dwarkesh Podcast, OpenAI researcher Noam Brown described multi-agent systems as a way to gain speed more than quality. Four agents solved tasks twice as fast and cost twice as much. At 16 agents, that pattern continued but became slightly less efficient.
Some work divides cleanly; some does not
The usefulness of parallel agents depends on whether a task can be split into separate pieces. Brown said web research and math can parallelize well, while writing a novel does not. In his example, assigning 10,000 agents to write a novel would be as pointless as assigning 10,000 people.
That distinction helps explain why adding agents can produce faster results without a matching improvement in the final output. If parts of a task can be handled at the same time, a team may reduce the time to completion. If the work depends on a unified thread of reasoning or coordination, more agents may create overhead without solving the core difficulty.
Brown said that scaling to very large numbers of agents remains largely unexplored because the costs are too high. That leaves an important practical limit: claims about large swarms should be weighed against their expense, especially when evidence of improved quality is modest.
The coordination tax shapes the trade-off
OpenAI developer Eric Provencher has warned against agent swarms for a related reason: coordination can break down between agents. He called the resulting burden the coordination tax. Even when individual agents can work in parallel, organizing their contributions may reduce the value of adding more.
For teams considering multi-agent systems, the findings point to a straightforward question: is faster completion valuable enough to justify the added token usage and coordination? On tasks that divide naturally, the answer may be yes. For work where agents struggle to coordinate or where quality scores barely move, a single agent may deliver a better balance of cost and performance.
The evidence does not support treating agent count as a simple measure of capability. Teams can buy speed, but whether they improve quality depends on the model, reasoning effort, and task. The Vals AI benchmark in particular suggests that running a team at maximum reasoning effort can raise costs without a corresponding advantage.