Why California's new AI law puts disclosure before testing

California Governor Gavin Newsom signed S.B. 53, the Transparency in Frontier Artificial Intelligence Act, into law. The measure requires large AI companies to publish safety protocols and report certain incidents, but it does not mandate safety testing or kill switches.

WTF Index TERMINATOR
◄ Terminator 2 Idiocracy 0 ►

The story concerns frontier AI safety regulation and notes weaker controls than advocates wanted, implying some unresolved risk from powerful systems.

Why California's new AI law puts disclosure before testing

California has chosen a lighter-touch path for regulating advanced AI systems. With S.B. 53, the Transparency in Frontier Artificial Intelligence Act, the state is asking major AI companies to show more of their safety work to the public and to state authorities, while leaving out the stricter testing and shutdown requirements that appeared in an earlier bill.

The result is a law with real reporting obligations, but a narrower safety mandate than many AI safety advocates had pushed for. It also lands in a state that houses 32 of the world’s top 50 AI companies, giving California AI regulation significance well beyond state borders.

What S.B. 53 Requires

On Monday, California Governor Gavin Newsom signed S.B. 53 into law. The law applies to AI companies with annual revenues of at least $500 million.

Those companies must publish their safety protocols on their websites. They must also report what the state calls “potential critical safety incidents” to California’s Office of Emergency Services.

The law includes whistleblower protections for employees who raise safety concerns. It also gives the attorney general authority to seek civil penalties of up to $1 million per violation when companies fail to comply with the reporting requirements.

The measure asks companies to explain how they use “national standards, international standards, and industry-consensus best practices” in AI development. But the law does not define those standards in detail, and it does not require independent verification of the companies’ claims.

How It Differs From S.B. 1047

S.B. 53 replaces Senator Scott Wiener’s earlier AI regulation effort, S.B. 1047. That previous bill would have required safety testing and “kill switches” for AI systems.

Governor Newsom vetoed S.B. 1047 last year after tech companies lobbied heavily against it. The new law keeps the focus on transparency, reporting, and internal safety descriptions rather than mandatory technical controls.

That shift matters because disclosure and testing solve different problems. Disclosure can make company practices more visible to the public, regulators, and employees. Mandatory testing would have required a more direct demonstration that systems met safety expectations before or during deployment.

Under S.B. 53, companies still have to describe their safety approaches. But the law stops short of forcing a particular safety test, a specific external review process, or a built-in shutdown mechanism.

What Counts As A Serious AI Incident

The law defines catastrophic risk narrowly. It covers incidents potentially causing 50+ deaths or $1 billion in damage through weapons assistance, autonomous criminal acts, or loss of control.

That definition sets a high threshold for the most severe category of concern. It also shows the law is aimed at extreme harms rather than every possible problem connected to AI systems.

The reporting structure centers on “potential critical safety incidents.” Companies covered by the law must report those incidents to California’s Office of Emergency Services. Employees who raise safety concerns receive whistleblower protections under the measure.

In practical terms, S.B. 53 creates a formal channel for serious AI safety information to reach state authorities. But it does not require companies to prove in advance that their systems cannot produce the harms described in the law.

Why California’s Choice Matters

California’s decision is important because the state is a central hub for AI development. According to the California state government, California houses 32 of the world’s top 50 AI companies. More than half of global venture capital funding for AI and machine learning startups went to Bay Area companies last year.

That means a state-level law can still affect companies building systems used around the world. It can also become a model, or a warning, for other governments considering how to regulate frontier AI.

Newsom framed the measure as a balance between protection and industry growth. “California has proven that we can establish regulations to protect our communities while also ensuring that the growing AI industry continues to thrive,” Newsom said in a statement.

The law also follows recommendations from AI experts convened by Newsom, including Stanford’s Fei-Fei Li and former California Supreme Court Justice Mariano-Florentino Cuéllar. Senator Wiener described the law as establishing “commonsense guardrails,” while Anthropic’s co-founder, Jack Clark, called the law’s safeguards “practical.”

The Limits Of A Disclosure-First Model

The central question is whether transparency alone can meaningfully reduce AI risk. S.B. 53 requires covered companies to publish safety protocols and report serious incidents, but the source article notes that the transparency requirements likely mirror practices already standard at major AI companies.

That could make the law easier for large companies to follow. It could also limit how much new protection the public receives if disclosures are broad, difficult to compare, or not backed by specific standards.

The political context also matters. According to The New York Times, Meta and venture capital firm Andreessen Horowitz have pledged up to $200 million to two separate super PACs supporting politicians friendly to the AI industry. Companies have also pushed for federal legislation that would preempt state AI rules.

S.B. 53 keeps one element from S.B. 1047: it creates CalCompute, a consortium within the Government Operations Agency, to develop a public computing cluster framework. The California Department of Technology will recommend annual updates to the law, though those recommendations require no legislative action.

For now, California’s AI law draws a clear line. It puts public safety disclosures, incident reporting, and whistleblower protections into law, while leaving mandatory safety testing and kill switches out. That makes S.B. 53 a significant AI governance step, but also a more industry-friendly one than the bill that came before it.