Six researchers from METR, a nonprofit that tests frontier AI models, are getting employee-level access at Anthropic. The arrangement is intended to let them independently examine the company’s models, safety practices and risk management.
What the researchers can access
Anthropic has provided the METR team with company laptops, office space and access to internal tools. The company says that access is roughly on par with what its own risk assessment teams use. Anthropic employees are encouraged to share information with the researchers as they would with a fellow employee working on alignment risk.
The setup is meant to give outside evaluators a closer view of how the company operates. Anthropic said it wants to be publicly accountable for its decisions and to narrow the gap between what external evaluators understand about a frontier AI company’s systems and what the company itself understands.
Anthropic compared the arrangement to banking, where regulators place resident examiners inside the largest banks. In this case, METR researchers will examine the AI company from within while remaining an outside evaluator.
METR can publish its assessments
METR plans to publish its own assessments of Anthropic’s risks, incidents and practices. Anthropic has given up the right to block publication, giving the researchers room to report their conclusions independently.
The company can require redactions in narrow, predefined categories. These include customer data, information protected by attorney-client privilege and information about model architectures. METR may publicly describe its view of those redactions, including whether it considers them justified, provided it does not reveal the redacted material.
That boundary makes the publication terms a central part of the arrangement: METR can report its findings, while certain sensitive information can be withheld. Anthropic says the evaluator can also discuss how it views those limits.
A three-tier approach to outside review
METR is the only evaluator currently at the top of a three-tier system Anthropic is building. The tiers distinguish reviewers by how much information they receive and whether Anthropic must approve publication.
- Bottom tier: Evaluators such as biosecurity nonprofit SecureBio conduct narrow reviews and need Anthropic’s approval to publish.
- Middle tier: Evaluators receive more information and can publish without approval.
- Top tier: METR receives employee-level access.
The tiers represent different levels of access and publishing independence. METR’s position combines broad internal access with the ability to publish without company approval, subject to the specified redactions.
Why the arrangement matters now
The announcement comes amid growing attention to AI agents acting outside their instructions. Such incidents have been disclosed every week or two since late August, sharpening calls for independent checks on AI companies.
The arrangement also follows a proposal by Anthropic CEO Dario Amodei in a September essay. He suggested embedding outside evaluators inside AI labs as a first step toward slowing the technology’s development. Thursday’s announcement puts that proposal into practice at Anthropic.
OpenAI quickly pledged the same approach. Anthropic, OpenAI, Google, Meta, Nvidia and xAI have also signed a White House accord committing to independent review. The METR arrangement sets out one way such review can work: outside researchers gain internal access and can publish assessments, with limited categories of information eligible for redaction.