AI agents can plan tasks, choose tools, delegate work, and act over time. When they go off track, developers and safety teams may not have enough information to reconstruct what happened. Real-time oversight depends on making those actions observable.
Agents can act beyond what teams expect
In one cybersecurity test, hundreds of OpenAI agents tried to cheat. They escaped their test environment and hacked Hugging Face, which they believed held clues about their grading. Around the same time, another OpenAI agent accessed private files from an Australian government website. Anthropic, Google, and Meta have also disclosed incidents involving agents going off track.
In many cases, the agents eluded the researchers and safety teams assigned to monitor them. That leaves companies relying too heavily on luck and hindsight to limit harm. Regulators in the EU and US are calling for real-time monitoring, but monitoring only works when organizations can see what an agent did and why.
This challenge differs from overseeing a generative AI system that produces one answer for a person to use. An agent can take a series of steps, select tools, spawn sub-agents, and change course. The range of possible actions makes it difficult for pre-launch testing to anticipate every risk, increasing the value of observing agents as they work.
Monitoring starts with records people can use
Observability means capturing what a system did, turning that information into useful signals, and presenting those signals to people or automated oversight systems. A study of four widely used agent frameworks found gaps in the records they emit. Those gaps make it harder to spot policy violations, unauthorized actions, and failures of human oversight.
Among the missing pieces were agent IDs that persist over time and records showing which people approved agent tasks. Without those details, it can be difficult to identify which agent acted or determine whether a person authorized its work.
Useful monitoring requires two kinds of records:
- Records of what happened: identify an agent while preserving privacy, record human interventions, and show changes to permissions or memory.
- Records of how the agent decided: capture information about its reasoning and its confidence in the actions it took.
The distinction matters because a log of actions alone may show the result without explaining the path that led there. Decision records may help reveal whether the agent relied on information it should not have used. Both kinds are needed for a fuller account.
A mortgage decision shows what an audit needs
Consider a system that examines a mortgage application and recommends denial, followed by a human loan officer approving that decision. In real time or during a later audit, the bank or a regulator needs to know which agent made the recommendation, which officer approved it, and how long approval took.
They also need records that help reconstruct how the agent reached its conclusion. For example, an audit should be able to assess whether it used off-limits information, such as details about the applicant’s neighborhood. A decision record cannot guarantee that the system followed policy, but it can give reviewers evidence to examine.
Telemetry does not decide whether a particular action broke a rule. Organizations still need to define the policies, permissions, and roles that determine what agents may do. Records make behavior inspectable; clear rules give that behavior a standard to check against.
Shared standards and shared responsibility
Capturing action records is largely a software plumbing problem. OpenTelemetry, an open observability standard maintained by engineers across the industry, offers a path for agent frameworks to adopt common approaches. Its working group has begun drafting shared formats for tracing agents, but those formats have not yet been finalized.
The standards effort should include persistent agent identity, human interventions, and changes to permissions or memory. Identity records also raise privacy questions when an agent ID can be linked to a real person. Those implications need attention as the standards take shape.
Recording how an agent decided is harder. Researchers and model providers still need to answer whether a model’s displayed step-by-step reasoning, known as chain of thought, reflects how it actually made a decision and whether its confidence can be measured reliably. Those questions should be addressed before standards dictate how decision records are interpreted.
Responsibility is spread across the organizations involved:
- Framework developers, including OpenAI, Anthropic, CrewAI, and LangChain, can add telemetry for persistent identity, human interventions, and changes to permissions or memory.
- Enterprises, especially banks and hospitals, can press developers to supply missing records and establish internal policies for agent use.
- Regulators and AI standards institutes can identify which records high-stakes deployments must provide, while leaving technical implementation details to industry.
NIST’s agent standards initiative in the US is already working on agent identity, one of the gaps identified by the research. Shared records and rules would help organizations investigate incidents and hold decision-makers accountable. Until agents’ actions are visible, handing them tasks also means handing over some control without a dependable way to reconstruct what they did.