Anthropic says Claude should not help anyone build a nuclear weapon. The company’s answer is a safety system developed with the Department of Energy (DOE) and the National Nuclear Security Administration (NNSA), after testing inside a secure government cloud environment.
The idea sounds direct: identify conversations that move toward dangerous nuclear territory and stop them before they produce harmful help. The harder question is whether that kind of safeguard can be evaluated clearly when the subject matter is classified, technically precise, and surrounded by uncertainty about what today’s AI models can actually do.
How the Claude nuclear safety work happened
At the end of August, Anthropic announced that Claude would not assist with building a nuclear weapon. According to Anthropic, the company partnered with the DOE and NNSA to make sure Claude would not reveal nuclear secrets.
The government side of the project used Amazon Web Services. AWS offers Top Secret cloud services to government clients, and the DOE already had several of these servers when it began working with Anthropic.
Marina Favaro, who oversees National Security Policy & Partnerships at Anthropic, told WIRED: "We deployed a then-frontier version of Claude in a Top Secret environment so that the NNSA could systematically test whether AI models could create or exacerbate nuclear risks." She added: "Since then, the NNSA has been red-teaming successive Claude models in their secure cloud environment and providing us with feedback."
That red-teaming process means the model was tested for weaknesses. Anthropic and nuclear experts then codeveloped what Favaro described as a nuclear classifier, a filter for AI conversations. The classifier was built from an NNSA-developed list of nuclear risk indicators, specific topics, and technical details that could show a conversation moving into harmful territory.
What the classifier is supposed to catch
The goal is not to block every discussion that mentions nuclear topics. Favaro said the system took months of tweaking and testing, and that it can catch concerning conversations without flagging legitimate discussions about nuclear energy or medical isotopes.
That distinction matters because the word “nuclear” can appear in many contexts. The same model may be asked about nuclear energy, medical isotopes, national security, weapons science, or historical nuclear research. A useful classifier has to separate ordinary or legitimate topics from requests that could create risk.
The underlying risk is complicated. Nuclear weapons manufacturing is a precise science and, as the source explains, a solved problem. Much of the information about America’s most advanced nuclear weapons is Top Secret, but the original nuclear science is 80 years old. North Korea proved that a dedicated country seeking the bomb could do it without a chatbot’s help.
That creates the central tension. If a determined country does not need Claude, the most serious concern may not be that a chatbot invents nuclear weapons knowledge from nothing. It may be that a capable model helps gather, summarize, or connect already available material in ways that reduce friction for a user with dangerous intent.
Why experts remain divided
Wendin Smith, the NNSA’s administrator and deputy undersecretary for counterterrorism and counterproliferation, told WIRED that AI-enabled technologies have shifted the national security space. Smith said NNSA’s expertise in radiological and nuclear security places it in a unique position to help deploy tools that guard against potential risk in these domains.
But both NNSA and Anthropic were vague about those potential risks. It remains unclear how helpful Claude or another chatbot would be in constructing a nuclear weapon.
Oliver Stephenson, an AI expert at the Federation of American Scientists, told WIRED: "I don’t dismiss these concerns, I think they are worth taking seriously." He added: "I don’t think the models in their current iteration are incredibly worrying in most cases, but I do think we don’t know where they’ll be in five years time … and it’s worth being prudent about that fact."
Stephenson also pointed to the problem of classification. When important details are hidden, outside observers cannot easily judge how much the classifier has improved safety. He said AI could potentially help synthesize information from different physics papers and publications on nuclear weapons, especially around detailed subjects such as implosion lenses.
His concern was not only about the tool itself. It was also about transparency. Stephenson said AI companies should be more specific about the risk model they are worried about, while acknowledging that government and industry collaboration can be useful.
The criticism: capability claims and trust
Heidy Khlaaf, the chief AI scientist at the AI Now Institute with a background in nuclear safety, was more skeptical. She described Anthropic’s claim that Claude will not help build a nuke as both a magic trick and security theater.
Khlaaf’s argument starts with training data. A large language model is only as good as the data it was trained on. If Claude did not have access to sensitive nuclear material, then testing whether it reveals such material may not prove much about the strength of the safeguard.
She told WIRED: "If the NNSA probed a model which was not trained on sensitive nuclear material, then their results are not an indication that their probing prompts were comprehensive, but that the model likely did not contain the data or training to demonstrate any sufficient nuclear capabilities." She argued that using an inconclusive result and common nuclear knowledge to build a classifier would be insufficient compared with legal and technical definitions of nuclear safeguarding.
Khlaaf also warned that announcements like this can feed speculation about abilities chatbots may not have. She questioned the assumption that Anthropic models would produce emergent nuclear capabilities without further training.
Another concern is access. Khlaaf said private AI companies are hungry for training data and suggested that the government’s broader interest in AI could give the industry access to sensitive national security data it could not get elsewhere. She asked whether private corporations should have access to such information, whether the topic is military systems, nuclear weapons, or nuclear energy.
What this means for AI safety
Anthropic disagrees with the criticism. A company spokesperson told WIRED that much of its safety work is focused on proactively building systems that can identify future risks and mitigate them. The spokesperson said the NNSA work helps Anthropic perform risk assessments and create safeguards against potential misuse.
The company is also offering its classifier to other AI companies. Favaro said Anthropic’s ideal outcome is a voluntary industry standard and a shared safety practice that others adopt. She said it would require a small technical investment and could meaningfully reduce risks in a sensitive national security domain.
The unresolved issue is proof. A classifier may be useful, but the public cannot see the full testing process, the controlled list, or the most sensitive details behind the risk model. That leaves readers with competing claims: Anthropic says it is preparing for future misuse, while critics argue the evidence may not show what the company suggests.
The debate is also a reminder that nuclear safety is not just about blocking the right words. It involves precision, verification, and human judgment. Khlaaf noted that large language models can fail at basic mathematics, and the source points to 1954, when a math error tripled the yield of a nuclear weapon the US tested in the Pacific Ocean.
Claude’s nuclear classifier may become a useful AI safety practice. But the larger question is still open: how should the public evaluate a safeguard when the risks are real, the technical details are partly classified, and the capabilities of future AI models remain uncertain?