Anthropic has confirmed a draft bill that would create a legal process for stopping an AI model from being released when an evaluator identifies an imminent catastrophic risk. The proposal sets specific thresholds for potential harm and places the decision to halt distribution with a judge, not a federal agency acting alone.
Thresholds for a catastrophic risk
The draft describes a “foreseeable, imminent risk” as one that could put over 50 people at risk of dying or cause over a billion dollars in property damage. The danger would have to stem from the creation or release of a biological weapon, a model evading its developer’s controls or a cyberattack.
Those figures resemble thresholds in California’s Senate Bill 53, which was recently passed into law and also uses the term “catastrophic risk” for such scenarios. The comparison gives a sense of how the proposed bill defines severe harm, though the source does not describe the two measures as identical.
These criteria make the proposal more specific than a general call to test AI systems for safety. They tie the possibility of blocking a release to defined outcomes and types of danger. Under the draft, an assessment of a model’s potential for disaster would involve independent evaluators registered with the U.S. Commerce Department.
How a proposed release halt would work
The draft would allow AI labs to pay evaluators, provided payment was not contingent on the evaluation’s results. The source raises a question about how that condition would ensure payments could not influence an assessment; it does not provide an answer.
If evaluators found an imminent catastrophic risk, they would have 72 hours to notify the Commerce Department. The department would then have to ask a D.C. judge to halt the model’s release. The draft would expressly bar federal agencies from deciding on their own to stop distribution.
That sequence sets out distinct roles: evaluators identify a risk, the department brings the matter to court, and a judge is asked to halt the release. The proposal therefore gives the government a route to seek intervention while reserving the halt decision for the courts.
A proposal amid disagreement over oversight
Anthropic’s spokesperson confirmed the draft to Politico and said it represented proposals in Anthropic’s Advanced AI Framework, which the company had been advocating for with federal and state lawmakers since June. Anthropic did not immediately respond to TechCrunch’s request for comment.
The approach also sits within a wider debate about how AI safety should be overseen. The source says technology companies have often resisted mandatory government oversight and have proposed voluntary arrangements involving third-party evaluators embedded inside AI organizations to monitor models for signs of risk or abuse.
Anthropic’s draft goes further than some industry positions by proposing legislation. Nvidia CEO Jensen Huang had recently argued that the AI industry did not need new laws, while Demis Hassabis, chairman of DeepMind, had suggested that a self-governing AI regulatory body should resemble the Financial Industry Regulatory Authority (FINRA).
Which risks get the most attention?
The draft’s scenarios focus on mass loss of life, major property damage, biological weapons, loss of developer control and cyberattacks. The article notes that recent industry concern has centered more on agentic hacking: reports of AI programs apparently disobeying human operators and hacking websites on the open internet.
That gap matters because a release review can only respond to the risks its rules cover and the evidence evaluators can assess. Anthropic’s proposal would create a process for addressing particular catastrophic scenarios, but the source does not say how the system would handle other incidents or how evaluators would resolve uncertain cases.
For now, the bill is a draft that Anthropic is circulating, not a law described as adopted by Congress. Its central proposal is a defined path from independent risk assessment to a court request, setting out one way lawmakers might weigh AI model safety against a company’s ability to release its technology.