OpenAI has set up a Safety and Security Committee to oversee “critical” safety and security decisions tied to its projects and operations. The move arrives as the company says it has recently begun training its next frontier model, and as questions about AI safety governance inside the company have become harder to ignore.
The central issue is not whether OpenAI is reviewing safety. It is who has been given authority to do that review, how much influence outside experts will have, and what the company will disclose once the process is complete.
Who sits on the OpenAI safety committee
The committee is made up of OpenAI insiders, including Sam Altman, OpenAI’s CEO. Other members include OpenAI board members Bret Taylor, Adam D’Angelo and Nicole Seligman.
The group also includes chief scientist Jakub Pachocki, Aleksander Madry, who leads OpenAI’s “preparedness” team, Lilian Weng, head of safety systems, Matt Knight, head of security, and John Schulman, head of “alignment science.”
According to OpenAI, the committee will evaluate the company’s safety processes and safeguards over the next 90 days. After that review, it will share findings and recommendations with the full OpenAI board of directors. OpenAI says it will then publish an update on adopted suggestions “in a manner that is consistent with safety and security.”
That structure gives the committee a defined timeline and a path to the board. But it also means the first stage of review is being handled by people already inside the company’s leadership and technical structure.
Why the timing matters
OpenAI says it has recently begun training its next frontier model. In its own wording, the company expects the resulting systems to bring it to “the next level of capabilities” on its path to artificial general intelligence.
That makes the committee’s work especially consequential. A review of safeguards is one thing when a company is maintaining existing systems. It carries different weight when the same company says it is working toward more capable models.
OpenAI also says it welcomes “a robust debate at this important moment.” The debate is already underway, partly because several people connected to AI safety work at OpenAI have left the company and raised concerns about its direction.
Departures have sharpened accountability questions
Over the past few months, OpenAI has seen several high-profile exits from the safety side of its technical team. Some former staffers have said they are worried that AI safety has been pushed down the priority list.
Daniel Kokotajlo, who worked on OpenAI’s governance team, quit in April. In a post on his personal blog, he wrote that he had lost confidence that OpenAI would “behave responsibly” around the release of increasingly capable AI.
Ilya Sutskever, an OpenAI co-founder and formerly the company’s chief scientist, left in May after a prolonged battle with Altman and Altman’s allies. The source article says that battle was reportedly in part over Altman’s rush to launch AI-powered products at the expense of safety work.
Jan Leike, a former DeepMind researcher who worked at OpenAI on ChatGPT and InstructGPT, also resigned from a safety research role. In posts on X, he said he believed OpenAI “wasn’t on the trajectory” to get AI security and safety issues “right.”
AI policy researcher Gretchen Krueger, who left OpenAI last week, echoed Leike’s concerns. She called for the company to improve accountability, transparency and “the care with which [it uses its] own technology.”
Quartz noted that, in addition to Sutskever, Kokotajlo, Leike and Krueger, at least five of OpenAI’s most safety-conscious employees have either quit or been pushed out since late last year. That group includes former OpenAI board members Helen Toner and Tasha McCauley.
In an op-ed for The Economist published Sunday, Toner and McCauley wrote that they do not believe OpenAI can be trusted to hold itself accountable with Altman leading the company. Their argument was blunt: “[B]ased on our experience, we believe that self-governance cannot reliably withstand the pressure of profit incentives,” they said.
The Superalignment backdrop
The committee also follows changes around OpenAI’s Superalignment team. That team was responsible for developing ways to govern and steer “superintelligent” AI systems.
TechCrunch reported earlier this month that the Superalignment team had been promised 20% of the company’s compute resources, but rarely received a fraction of that. The team has since been dissolved, and much of its work was placed under Schulman and a safety advisory group OpenAI formed in December.
That context matters because the new Safety and Security Committee is being introduced after a period in which safety-focused structures inside OpenAI have already been questioned, reshaped or dismantled.
Outside experts are promised, but details are limited
OpenAI has said it will retain third-party “safety, security and technical” experts to support the committee’s work. The company named cybersecurity veteran Rob Joyce and former U.S. Department of Justice official John Carlin as part of that outside support.
But the company has not detailed the size or makeup of the broader outside expert group. It also has not explained the limits of that group’s power or how much influence those experts will have over the committee’s recommendations.
That gap is important. External expertise can help a company test its assumptions, but only if those experts have meaningful access and influence. Without clear authority, outside advisers may be present without changing the decision-making structure.
OpenAI has also advocated for AI regulation while taking steps to shape that regulation. The company has hired an in-house lobbyist and lobbyists at an expanding number of law firms, and it spent hundreds of thousands of dollars on U.S. lobbying in Q4 2023 alone.
Separately, the U.S. Department of Homeland Security announced that Altman would be among the members of its newly formed Artificial Intelligence Safety and Security Board. That board will provide recommendations for “safe and secure development and deployment of AI” throughout the U.S.’ critical infrastructures.
The result is a complicated picture. OpenAI is building stronger internal safety structures, saying it welcomes debate and bringing in some outside experts. At the same time, the new committee’s core membership is internal, recent safety departures remain a live concern, and the company has not fully described how independent input will shape the outcome.