AI agents are moving deeper into software security. New research from UC Berkeley shows that advanced AI models can do more than write code: they can analyze codebases, run tests, build proof-of-concept exploits, and uncover real vulnerabilities.
The finding matters because the same capability can help defenders and attackers. Better automated bug discovery could make software safer, but it could also lower the barrier for finding flaws that can be used against live systems.
CyberGym put AI agents inside real codebases
AI researchers at UC Berkeley tested how well current AI models and cybersecurity agents could find vulnerabilities in 188 large open source codebases. The benchmark used for the work is called CyberGym.
The researchers tested frontier AI models from OpenAI, Google, and Anthropic, along with open source models from Meta, DeepSeek, and Alibaba. They combined those models with bug-finding agents including OpenHands, Cybench, and EnIGMA.
The setup was designed to measure more than simple code reading. The agents were asked to analyze new codebases, run tests, and craft proof-of-concept exploits. The researchers also gave them descriptions of known software vulnerabilities from the 188 software projects and checked whether the agents could independently identify the same flaws.
They were then asked to hunt for new vulnerabilities on their own. Through that process, the AI tools generated hundreds of proof-of-concept exploits.
The results show real progress, not full automation
The AI models identified 17 new bugs. Of those, 15 were previously unknown, or zero-day, vulnerabilities. Two vulnerabilities had already been disclosed and patched.
Dawn Song, a professor at UC Berkeley who led the work, described the result as significant. “Many of these vulnerabilities are critical,” she says. She also says the outcome reflected a shift in what AI systems can do when coding skill is combined with stronger reasoning. “This is a pivotal moment,” she says. “It actually exceeded our general expectations.”
Song also said the researchers did not push the agents as far as they might have. “We didn't even try that hard,” Song says. “If we ramped up on the budget, allowed the agents to run for longer, they could do even better.”
The implication is clear: as AI agents improve, vulnerability discovery may become more automated. That could help companies find weaknesses faster. It could also help hackers identify and exploit flaws more easily.
Zero-day discovery is the central concern
Zero-day vulnerabilities are especially important because they are previously unknown and may offer a path into live systems. The UC Berkeley work adds to growing evidence that AI can help automate discovery of these flaws.
The article points to other examples. Security expert Sean Heelan recently discovered a zero-day flaw in the widely used Linux kernel with help from OpenAI’s reasoning model o3. Last November, Google announced that it had discovered a previously unknown software vulnerability using AI through a program called Project Zero.
AI is also becoming visible in bug hunting more broadly. An AI tool from startup Xbow has climbed HackerOne’s leaderboard for bug hunting and currently sits in top place. Xbow also recently announced $75 million in new funding.
Brendan Dolan-Gavitt, an associate professor at New York University Tandon and a researcher at Xbow, says the UC Berkeley work shows realistic zero-day discovery across a relatively large amount of code using a wide range of AI-powered tasks.
He expects AI to drive more attacks involving zero-day exploits. “That's rare right now, because there are very few people who have the expertise to find new vulnerabilities and build exploits for them,” he says.
Experts warn against replacing human judgment
The research also shows that AI security tools remain limited. The systems were unable to find most flaws and struggled with especially complex ones.
Katie Moussouris, founder and CEO of Luta Security, says the work is valuable partly because it shows AI is not yet a replacement for human expertise. The best model and agent combination, Claude and OpenHands, found around 2 percent of the vulnerabilities.
“Don’t replace your human bug hunters yet,” Moussouris says.
Her concern is not only that AI might be used to hack software. She says she is less worried about that than about companies putting too much investment into AI while neglecting other techniques.
That is an important distinction. AI agents may become useful additions to security programs, but the source evidence does not show that they can handle the full job on their own. Their value appears strongest when they expand testing and discovery capacity, not when they replace expert review.
Responsible disclosure becomes more important
Hayden Smith, a cofounder of Hunted Labs, says agentic tools are compelling for zero-day discovery. Hunted Labs provides tools, including some that use AI, for analyzing code for weaknesses.
Smith adds that if AI makes vulnerability discovery possible for more people, responsible disclosure will become more important. That point follows directly from the dual-use nature of the technology: the same finding can be reported so it can be fixed, or used before defenders know it exists.
UC Berkeley researchers have also studied how AI models perform on bugs connected to bug-bounty rewards. In work posted online in May, Song and other researchers found that these tools could potentially earn tens of thousands of dollars.
Claude Code, from Anthropic, was the most successful in that effort. It found bugs worth $1,350 on bug bounty boards and designed patches for vulnerabilities worth $13,862 for a cost of a few hundred dollars in API calls.
In a blog post in April, Song and several other AI security experts warned that improving models are likely to benefit attackers over defenders in the near future. Song and other researchers have also established the AI Frontiers CyberSecurity Observatory, a collaborative effort that will track the capabilities of different AI models and tools through several benchmarks.
The practical message is measured but serious. AI agents are already finding real software bugs, including zero-days. They are not yet a substitute for skilled security teams, but they are becoming powerful enough that defenders need to track them closely.