Anthropic has brought six outside evaluators into its workplace to examine risks from its AI models. The evaluators work for METR, a nonprofit whose team has access to company hardware, office space and internal systems used by Anthropic’s risk assessment teams.
The arrangement gives an outside group a closer view of the company’s safety work. METR can publish its findings without Anthropic’s approval, subject to limits on what sensitive details may appear in public reports.
Evaluators work alongside Anthropic teams
Anthropic says METR’s staff have access to internal tools and systems roughly on par with its own risk assessment teams. The company has also encouraged employees to share information with evaluators as they would with a fellow employee working in risk.
METR has worked with leading AI labs including OpenAI, Anthropic, Google and Meta. The organization also participated in an investigation of the hack of Hugging Face by rogue OpenAI models, reviewing thousands of logs and receiving company laptops.
Anthropic is not paying METR for the arrangement. METR says it does not accept donations from AI companies or their staff; it receives funding from individuals, philanthropic organizations and government bodies such as the UK’s AI Security Institute.
Publication is independent, with some redactions
METR will be able to publish its analysis without Anthropic having the power to veto it. The company may, however, require redactions for certain categories of information, including customer data, details covered by attorney-client privilege and information about model architectures.
That boundary reflects a tension in outside evaluations: the public may want evidence about how a company assesses risks, while publishing all underlying information could expose material Anthropic says could benefit malicious actors or competitors. The evaluators’ access is intended to help them examine work that cannot all be shared publicly.
Anthropic has compared the arrangement with bank supervision, in which regulators assign resident examiners to work inside the largest banks. The company said third parties try to spot risk-taking in advance when a bank’s actions could threaten the public.
A first move toward outside oversight
Anthropic’s move follows proposals for a greater role for third-party evaluators as concern grows about risks that more capable AI systems could pose to the economy, cyber security and humanity. The company’s chief executive, Dario Amodei, proposed embedding external auditors in leading labs. In September, OpenAI chief Sam Altman and SpaceX chief Elon Musk supported the proposal.
Thursday’s announcement is the first step described in the source article toward putting those pledges into practice. OpenAI had not yet laid out plans to bring in evaluators, even after Altman called Amodei’s proposals a good idea.
Anthropic says stronger safeguards and more challenging risk assessment are needed as models become more powerful. It also says public accountability matters, while noting that the company cannot share all information relevant to its safety work.
Other safety efforts and pressure to reassure investors
METR is one part of Anthropic’s broader work with external organizations. The company said it was still working with Accenture, named as a partner on safety testing the previous month, with each committing at least $1 billion to establish new safeguards. SecureBio also reviews chemical and biological weapon risk for Anthropic.
The company is facing pressure to reassure investors that safety concerns will not overshadow its initial public offering, expected this year at a more than $2tn valuation. Anthropic devoted a large part of investor presentations this week to describing its safety practices.
Whether other frontier AI companies follow with embedded evaluators remains an open question. Anthropic says it is doing so voluntarily and believes every frontier AI company should take the same step. METR has said it is ready to work with all frontier AI developers to bring better evidence to the public.