New York pushes AI giants toward public safety rules

New York's RAISE Act would require major AI developers to publish safety protocols and risk assessments before making frontier AI models publicly available in the state. The bill has passed the state Senate and now depends on a decision from Governor Kathy Hochul.

WTF Index TERMINATOR
◄ Terminator 2 Idiocracy 0 ►

The story centers on regulation meant to reduce risks from powerful frontier AI systems and dangerous behavior.

New York pushes AI giants toward public safety rules

New York is moving toward a state-level framework for advanced AI oversight, with a bill that would place new disclosure and incident-reporting duties on the largest AI developers before they release powerful models to the public.

The Responsible AI Safety and Education (RAISE) Act has already passed the state Senate. It now awaits a decision from Governor Kathy Hochul, and would only become law if signed by the governor.

What the RAISE Act would require

If enacted, the RAISE Act would apply to major AI developers making so-called "frontier AI" models publicly available in New York. The core requirement is straightforward: companies would need to publish safety protocols and risk assessments before release.

The bill is aimed at the risks associated with advanced models, not routine software products. The source describes the law as focused on strict safety measures intended to reduce risks from powerful systems.

The proposal would also require companies to report serious incidents. The examples given are model theft and dangerous AI behavior. Violations could bring civil penalties reaching up to $30 million.

In practical terms, the bill would shift some AI safety work from private internal process into a more visible public and regulatory posture. Developers covered by the law would need to show, before public availability in New York, that they have assessed risk and defined safety procedures.

Why the bill targets only the largest developers

Democratic Senator Andrew Gounardes, one of the bill's sponsors, has emphasized that the measure is not designed for every AI lab, startup, or research group. According to the source, it would apply only to large companies that spend more than $100 million on model training.

Startups and academic institutions would be exempt. That threshold is central to the bill's political argument: the companies with the greatest resources and the most advanced systems would carry the compliance burden, while smaller actors would not be pulled into the same framework.

Gounardes told TechCrunch that the aim is to balance safety and innovation. He put the urgency in direct terms: "The window to put guardrails in place is closing fast," he said.

The bill was designed by Gounardes together with Assemblymember Alex Bores. The source says they shaped it to avoid problems associated with earlier efforts, especially California's failed SB 1047 bill.

Two limits are especially important. The RAISE Act does not require mandatory kill switches. It also does not hold companies liable for models that are later modified.

How the tech industry is responding

Even with those limits, the proposal has met resistance from Silicon Valley. Andreessen Horowitz partner Anjney Midha called the bill "dumb" on X and warned that it could put the U.S. at a competitive disadvantage.

Jack Clark, co-founder of Anthropic, has also raised concerns about how the bill could affect smaller companies. Gounardes has rejected that criticism by pointing back to the bill's narrow target: it is meant for the largest players, not smaller organizations.

Major companies named in the source include OpenAI, Google, Anthropic, and Meta. OpenAI, Google, and Meta have so far not commented publicly on the bill.

One criticism is that companies could decide not to offer their models in New York at all. Bores considers that unlikely. His reasoning is economic: New York is the third-largest economy in the U.S., so withdrawing from the state would make little economic sense.

The federal fight could change the path

The RAISE Act is also unfolding against a broader conflict over whether AI regulation should happen at the state level or be handled through unified federal rules.

A few weeks ago, the U.S. House of Representatives passed a legislative package that would impose a ten-year ban on state-level AI regulations. If the so-called "One Big, Beautiful" bill also passes the Senate, laws like the RAISE Act would be blocked in the future.

Supporters of the moratorium include major tech companies and free-market think tanks. Their argument is that AI developers need unified federal rules rather than a patchwork of state requirements.

Critics take the opposite view. They warn that such a moratorium could weaken consumer protections and shift the playing field toward industry interests.

That makes New York's bill more than a local AI policy dispute. It is a test of whether states can move first on frontier AI rules, or whether future regulation will be delayed or centralized by federal action.

What is at stake now

The immediate question is whether Governor Kathy Hochul signs the RAISE Act. Until that happens, the bill is not law.

If it is enacted, New York would become one of the first places in the United States to impose this kind of safety protocol and risk assessment requirement on advanced AI models. The law would not ban frontier AI models. It would require covered developers to disclose more about how they evaluate and manage risk before making those models publicly available in the state.

The central tradeoff is clear from the debate. Supporters see the bill as a targeted guardrail for the largest AI developers. Critics see a risk to competitiveness, market access, or smaller-company activity. The outcome now depends on the governor's decision and on whether federal lawmakers move ahead with a ten-year block on state-level AI regulation.