AI companies use benchmarks to show how their models perform, but a score is only useful if people can reproduce it and understand what it measures. Irene Solaiman, Hugging Face’s chief policy officer, says those conditions are not yet reliably met across the industry.
Scores are hard to reproduce and compare
Benchmarking is meant to offer a neutral way to compare AI products. Yet the testing methods are complicated and still developing, Solaiman said during a discussion at TechCrunch Disrupt last week. She argued that evaluations need to become a more consistent science if the industry is to depend on them.
Reproducing a result is one part of assessing whether it can be trusted. Solaiman said that access to the model matters, including access to its open weights, because evaluators need enough information to run the test again.
Even when a single evaluation is credible on its own, it may not be easy to compare its result with another test. Solaiman described evaluations as an evolving science, saying the industry does not yet know how to compare them meaningfully.
That gap limits what benchmark scores can tell buyers, researchers and AI companies. A result may indicate how a model performed under one test’s conditions, while offering little guidance about how it stacks up against results from different evaluations.
Third-party evaluators need clear qualifications
Interest has grown in embedding supposedly neutral third-party evaluators within AI organizations. The idea has gained attention as AI labs face safety concerns following incidents involving their technology. The source notes that Hugging Face was among the organizations targeted by one such incident.
Labs have promoted third-party evaluation as an alternative to active government regulation for identifying and reducing model safety concerns. But the arrangement raises questions about whether evaluators will be independent and effective, and whether they have the expertise to assess the work.
Anthropic’s decision to hire Accenture as its first evaluator drew backlash, with critics questioning the consulting firm’s qualifications. Solaiman said an evaluator ecosystem is in demand, but standards for deciding who can assess which tasks need more consistency.
Those standards matter because the evaluator’s role depends on what is being tested. A person or organization qualified to review one kind of model behavior may not necessarily be suited to assess another task or the complete system a user encounters.
Testing now includes the tools around a model
Evaluations are expanding beyond the model itself to include the surrounding system, including the harness and the agent. A harness is the tool a user uses to access a model, and Solaiman said it can affect performance substantially. A weaker model paired with a better harness could outperform a frontier model.
This means a model-only score may not capture the experience or capability that people actually use. Testing the agent as a whole can account for more of that system, but Solaiman said agentic evaluations are difficult and expensive.
She said there are more evaluations for models than for harnesses, while better ways to assess the holistic agent are being developed. The distinction matters because users often get value from the complete agent, rather than from a model considered in isolation.
Consistency could make benchmarks more useful
For benchmarking to remain useful to industry leaders, the tests need to be easier to reproduce and compare. A shared approach could help clarify what each score represents, what parts of a system were included, and whether different evaluators are applying comparable standards.
Solaiman’s account points to a field still catching up with the products it evaluates. Model tests, harness assessments and whole-agent evaluations each pose different challenges. Without clearer methods and qualifications, benchmark results may offer an incomplete basis for comparing AI systems or judging their safety.