AI companies are trying to stop their systems from becoming tools for malicious hackers. But the same restrictions meant to prevent abuse are creating friction for people whose job is to find weaknesses before criminals do.
Cybersecurity researchers described a recurring problem: the same prompt can be part of defense or offense, depending on context. When a model refuses to engage, the refusal can block a real security review as easily as it can block harmful activity.
Guardrails are becoming part of the security workflow
For months, major AI companies have built vetted programs and stricter model behavior around cybersecurity tasks. Anthropic and OpenAI both offer programs that let approved cybersecurity researchers use models with fewer restrictions: OpenAI's Trusted Access for Cyber program and Anthropic's Cyber Verification Program.
The approach reflects a hard problem. AI models can help analyze code, understand software behavior, and reason about vulnerabilities. They can also help users build and execute attacks if the same capabilities are misused.
That tension became more visible in June, when the U.S. government placed export control restrictions on Anthropic's AI models Mythos and Fable. The move was prompted at least in part by a report claiming the models' guardrails could be bypassed to build and execute malicious cyberattacks.
The restrictions on Fable 5 and Mythos 5 were later lifted. Fable 5 returned to general access on July 1, while Mythos 5 was reintroduced only to vetted U.S. organizations as part of the government's review process.
The same tool can defend and attack
Researchers who work on offensive cybersecurity say the line between useful defensive work and dangerous offensive work is not clean. Offensive researchers proactively test systems for weaknesses. In many cases, proving that a flaw can be exploited is part of showing that it is serious enough to fix.
Chris Anley, chief scientist at NCC Group, said asking a model to try to exploit a bug can be an important step in confirming whether a vulnerability is real. If the model simply refuses, the guardrail can get in the way of defense.
Anley explained that a prompt such as asking a model to fix code can serve two purposes at once. It can help defenders understand and repair a weakness, but it can also reveal how someone might find serious vulnerabilities in the same code base.
His point is that AI security tooling is not easily divided into clean categories. The capability that helps a researcher validate a fix may also help another person understand how to attack. That makes rigid model refusals difficult to tune without harming legitimate work.
Researchers are adapting in different ways
Several people in offensive cybersecurity described different levels of reliance on AI tools. Some use frontier models for parts of the workflow where the guardrails are less likely to interfere. Others avoid cloud-based models for sensitive vulnerability discovery.
Paolo Stagno, chief technology officer at Crowdfense, said his team uses frontier models only for reverse engineering. For finding vulnerabilities or building exploits, he said they avoid cloud-based models because sensitive vulnerability data could leak or be used in future training runs.
For those tasks, Stagno said they use open source models locally, because that setup does not require sharing data outside the model. This is not only about restrictions. It is also about control over sensitive research material.
Giuseppe Cali, a security researcher who finds zero-days and develops exploits, said guardrails are not blocking his work because he does not use AI for offensive work. He uses AI for initial reverse engineering, understanding code, and building supporting tools.
Cali said AI tools can accelerate that preparation work and let him focus on vulnerability discovery. But he still wants to handle actual bug discovery and weaponization himself.
Open source models are becoming the fallback
When guardrails stop a task, some researchers move to open source AI models with no guardrails at all. Anley said he and colleagues sometimes fall back on open source models when they hit those roadblocks.
One researcher at a smartphone-component manufacturer, speaking anonymously because he was not authorized to talk to the press, said his employer is not part of Anthropic's CVP program. As a result, he said the tools are barely useful for vulnerability work because the guardrails are too strict.
Chris Thompson, chief executive of RemoteThreat and founder of Offensive AI Con, said guardrails can be inconsistent even inside vetted programs from Anthropic and OpenAI. In his view, researchers can spend time negotiating with the model instead of analyzing a vulnerability and reasoning through exploitability.
Thompson said this pushes responsible researchers toward Chinese open source models like GLM, which can be downloaded and run locally without vetting or usage restrictions. His concern is that restrictions on U.S.-governed systems may drive legitimate work elsewhere.
The access debate is not settled
The core dispute is not whether malicious use should be blocked. The question is who decides which security work is acceptable, and how those decisions are implemented in tools that researchers increasingly want to use.
Mark Dowd, a well-known security researcher, criticized the role of major AI companies in making those calls. He said, “it's not really comfortable to me that these random large companies are making arbitrary decisions about what is safe in security and what's not.”
Thompson argued that frontier AI labs should open up their programs, provide responsible access, and hold abusers accountable. He warned that security firms and legitimate researchers are being restricted while attacks are expected to move at speed and scale.
For now, AI guardrails remain a practical obstacle for some offensive cybersecurity researchers and a non-issue for others. The pattern is clear, though: when approved access, cloud privacy concerns, or refusal behavior become too burdensome, researchers have alternatives. Those alternatives are often local, open source, and outside the vetted systems AI companies are trying to build.