A new California AI policy report argues that lawmakers should not wait for every serious AI danger to show up in practice before writing rules. The central idea is simple: frontier AI systems may create risks that are still uncertain, but uncertainty does not make those risks irrelevant.
The 41-page interim report comes from the Joint California Policy Working Group on AI Frontier Models, an effort organized by Governor Gavin Newsom after his veto of California's AI safety bill, SB 1047. The final version is due out in June 2025.
Why the report matters now
The report arrives after a difficult period for AI safety policy in California. Newsom vetoed SB 1047, saying the bill missed the mark, while also acknowledging the need for a more extensive assessment of AI risks to guide legislators.
That background makes the new report important. It does not try to revive one bill directly, and it does not endorse specific legislation. Instead, it lays out a framework for how lawmakers might think about frontier AI models and the companies building them.
The report was written by Fei-Fei Li, Jennifer Chayes (UC Berkeley College of Computing dean), and Mariano-Florentino Cuéllar (Carnegie Endowment for International Peace president). Before publication, it was reviewed by industry stakeholders from across the AI policy debate, including Yoshua Bengio and Ion Stoica.
Future risks are part of the policy question
The report's most direct message is that AI safety laws should cover more than harms already documented in the world. It says policymakers should consider risks that "have not yet been observed in the world" when designing AI regulatory policies.
That does not mean the authors claim every extreme AI scenario is proven. The report says there is an "inconclusive level of evidence" for AI's potential to help carry out cyberattacks, create biological weapons, or produce other extreme threats.
But the authors argue that lack of certainty is not the same as lack of responsibility. Their reasoning is that some risks can be serious enough that policymakers should plan for them before they become visible. The report uses the example of a nuclear weapon to make that point: society does not need to observe one exploding to understand that it could cause extensive harm.
This is the key tension for AI governance. If lawmakers regulate only what is already happening, they may move too slowly for systems that can change quickly. If they regulate too broadly, they risk missing the practical details that determine whether rules actually work.
Transparency is the main tool
The report focuses heavily on transparency around frontier AI model development. It suggests that laws may need to require AI model developers to publicly report important information about how their systems are tested, secured, and built.
According to the report, public reporting could include:
- safety tests run by AI model developers;
- data-acquisition practices;
- security measures;
- corporate policies tied to AI risk;
- areas of public concern raised by employees or contractors.
The report also calls for stronger standards around third-party evaluations. That matters because company self-reporting alone can leave lawmakers and the public dependent on claims made by the same organizations developing the models.
The phrase that captures the report's approach is trust but verify. Developers and employees should have channels to share information about public concerns, while testing claims should also be checked through third-party verification.
Whistleblowers and outside checks
The report also supports expanded whistleblower protections for AI company employees and contractors. That recommendation fits the broader transparency theme: people inside frontier AI labs may see safety testing, security practices, or internal policies that are not visible from outside.
In practical terms, whistleblower protections could make it easier for workers to raise concerns without relying only on internal company processes. The report does not frame this as a replacement for formal testing or external review. It presents it as another route for information to reach the public-interest side of the policy system.
This is especially relevant because frontier AI labs such as OpenAI are building systems whose risks may be difficult for outsiders to evaluate. If lawmakers want more visibility into model development, they need both structured reporting and credible ways to test or challenge what companies say.
How it connects to SB 1047 and SB 53
The report appears to align with several components of SB 1047 and Scott Wiener's follow-up bill, SB 53. One shared idea is requiring AI model developers to report the results of safety tests.
At the same time, the report has been received positively by figures on different sides of the AI policymaking debate. Dean Ball, an AI-focused research fellow at George Mason University who was critical of SB 1047, described it on X as a promising step for California's AI safety regulation. Wiener, who introduced SB 1047 last year, said in a press release that it builds on urgent conversations around AI governance that began in the legislature in 2024.
That reception is notable because AI safety policy has often split supporters of stronger rules from critics who see some proposals as poorly designed. The report's emphasis on transparency, verification, and future risk may give California lawmakers a less rigid starting point.
The larger implication is that the next phase of AI safety laws may be less about proving that every feared harm has already occurred and more about deciding what evidence, reporting, and oversight are needed before risks become harder to manage.