California’s debate over artificial intelligence regulation has reached a decisive stage. SB-1047, a contentious AI safety bill focused on large artificial intelligence models, has passed the California State Assembly by a 45–11 vote and is now close to Governor Gavin Newsom’s desk.
The measure has drawn support from major AI figures who argue that powerful systems need enforceable safeguards. It has also triggered warnings from critics who say the bill could push developers into defensive behavior, make open-source work riskier, and regulate model development instead of harmful uses.
What SB-1047 Would Require
At the center of SB-1047 is a requirement aimed at creators of large artificial intelligence models. The bill asks those creators to build in a “kill switch” that could be used if a model begins creating “novel threats to public safety and security.”
The concern is especially focused on systems acting “with limited human oversight, intervention, or supervision.” In plain terms, the bill is about placing a hard stop inside certain advanced AI systems before they can move beyond acceptable safety boundaries.
Supporters frame this as a basic protection for frontier AI models. The source article notes that the bill is focused on large models, including those that cost over $100 million to train, rather than smaller AI projects.
That distinction matters to the bill’s defenders because it narrows the target. Their case is that the most capable and expensive models should carry more safety obligations, while smaller startups should not face the same compliance burden.
Why Supporters Say Regulation Cannot Wait
Bill sponsor and state senator Scott Weiner pointed to backing from AI industry luminaries after the legislative passage Wednesday. Among those cited were Geoffrey Hinton and Yoshua Bengio, who both last year also signed a statement warning of a “risk of extinction” from fast-developing AI tech.
Bengio has argued publicly that SB-1047 sets a basic floor for regulation. In a recently published editorial in Fortune magazine, he described the bill as outlining a minimum standard for effective oversight of frontier AI models.
“We cannot let corporations grade their own homework and simply put out nice-sounding assurances,” Bengio wrote. “We don’t accept this in other technologies such as pharmaceuticals, aerospace, and food safety. Why should AI be treated differently?”
The core argument from this side is simple: if AI developers are building systems with public safety implications, then voluntary promises are not enough. Supporters want enforceable duties before a high-risk system creates damage that is difficult to reverse.
They also see California as a meaningful place for this debate because the state is deeply tied to the AI industry. The bill’s progress through the legislature suggests that many lawmakers are willing to act even while the technology and its risks are still being argued over.
Why Critics Want a Veto
Opponents say SB-1047 is aimed at the wrong problem. Some critics argue that the bill focuses on dramatic future risks from imagined AI systems while doing less to address current AI harms such as deep fakes or misinformation.
Stanford computer science professor and AI expert Fei-Fei Li made a broader argument against the bill in a separate Fortune editorial from earlier this month. She called the legislation “well-meaning” but warned that it will “have significant unintended consequences, not just for California but for the entire country.”
One key concern is liability. Li argued that making the original developer responsible for modified versions of a model would “force developers to pull back and act defensively.” According to her, that could limit open-source sharing of AI weights and models and create a significant impact on academic research.
California business leaders have also urged Newsom to reject the bill. In an open letter Wednesday, they called SB-1047 “fundamentally flawed” and said it improperly “regulates model development instead of misuse.”
The group warned that the measure would “introduce burdensome compliance costs” and “chill investment and innovation through regulatory ambiguity.” Their position is that uncertainty around obligations and liability could make AI companies less willing to build, release, share, or invest in new systems.
The Decision Now Moves To Newsom
SB-1047 has already cleared major legislative hurdles. It passed the Assembly by a 45–11 vote after a 32–1 state Senate vote in May. The bill still faces one more procedural state senate vote before it goes to Governor Gavin Newsom.
If the Senate confirms the Assembly version as expected, Newsom will have until September 30 to decide whether to sign the bill into law. If he vetoes it, the legislature could override that veto with a two-thirds vote in each chamber.
The source article notes that such an override is a strong possibility because the bill has already received overwhelming support in both chambers. That means Newsom’s choice may not be the final word, but it is still the next major test for the future of AI safety regulation in California.
Newsom has already described the tension he sees. At a UC Berkeley Symposium in May, he said he worried that “if we over-regulate, if we overindulge, if we chase a shiny object, we could put ourselves in a perilous position.”
At the same time, he acknowledged that the warnings coming from AI leaders change the political environment. He said that when the people who helped create the technology are saying, “Help, you need to regulate us,” the situation looks different.
What The Fight Reveals About AI Policy
The SB-1047 debate is not only about one California bill. It shows the central conflict in AI policy: how to act early enough to reduce serious risks without freezing research, open-source collaboration, and investment.
Supporters see a narrow safety rule for the largest models. Critics see a broad liability and compliance framework that could reshape how AI is developed and shared.
Both sides are arguing from future consequences. One side worries that advanced models could create public safety and security threats if developers are left to self-police. The other worries that unclear regulation could slow useful work, weaken research, and make developers less open.
Newsom’s decision will therefore carry weight beyond the text of SB-1047. Whether he signs or vetoes the bill, California’s fight over AI safety, innovation, open-source models, and regulatory responsibility is now one of the clearest signals of how governments may try to manage the next stage of artificial intelligence.