AI agents can complete complex tasks, but current benchmarks and company simulations suggest they still make enough mistakes to require close supervision. Plurai, a startup founded by former autonomous-driving technologists, wants to improve reliability by testing agents against simulated versions of the systems and policies they will encounter at work.
Why workplace agents need more testing
Benchmarks show that agent performance can fall short of what is needed for independent work. LiveBench 2.0 found Claude Fable 5.2 and ChatGPT-6 Astra succeeded on 82.9 and 75.6 out of 100 tasks, respectively. In a simulated corporate environment, AI infrastructure company EON found that its strongest tested model, Claude Opus 4.8, completed tasks with 76.8% accuracy.
Those results point to a gap between a convincing demonstration and dependable work. An agent may handle a clear request, yet struggle when a task involves company rules, several information sources, or a long exchange with a customer.
For example, an agent booking travel might disregard spending policies when a customer presses it. An agent that can answer a question from one database may falter when it has to combine information across multiple data sets. In customer service, a lengthy conversation can fill the agent’s available context and send it down the wrong path.
Plurai’s simulation approach
Plurai builds a digital twin of a company’s electronic systems, including the knowledge base and tools available to an agent in production. It maps potential paths an agent might take through those systems, then uses the resulting graph to evaluate the agent and prepare fixes for edge cases.
The idea draws on a method used in autonomous driving: create simulated environments and use synthetic data to prepare systems for difficult situations. Plurai’s founders, Ilan Kadar and Elad Levi, previously worked at Nexar, an autonomous-driving technology company. Kadar also worked with Waymo on simulating and evaluating its end-to-end deep learning driving software.
Kadar told TechCrunch that the founders recognized a familiar pattern in AI agents: building a demo is easy, while making a system reliable and cost effective is harder. Plurai’s platform is designed to work alongside tools such as Claude Code, Codex, Copilot and Cursor.
From evaluation to smaller models
Plurai’s larger ambition is to use the information gathered through simulation to post-train small, open-weight models for individual customers. The company says tests conducted with Nvidia and Nebius showed its models outperforming mid-tier frontier models at lower costs.
The process, as Kadar described it, starts with a large language model and a task. Simulation helps break the task into smaller parts, which can then be used to train an open-weight model. The simulation can also check whether the resulting model delivers quality that is higher than or comparable to the frontier model, while cutting costs by an order or two orders of magnitude, he said.
This approach ties model training to a company’s own workflows and information. Rather than relying only on a broadly capable model, a business could test and tune a smaller one against the situations its agents are expected to handle.
Company-specific knowledge is the test
Reah Miyara, a Google DeepMind researcher and Plurai angel investor, said the startup’s advantage comes from understanding failures in production and generating synthetic data to tune models. She pointed to a business concern: frontier-capable models may not fit a company’s specific data, while their costs may not be sustainable over time.
Plurai is also betting that lessons from AI in physical settings can apply to agents working in digital systems. General AI models are giving robots new capabilities, but the source says they are not yet reliable enough for broad deployment. Robots designed for specific tasks, by contrast, are already working for customers and collecting real-world data.
Kadar argues that even more capable general models will not automatically understand a particular business or its policies. Plurai’s proposal is to use a company’s systems as the setting for repeated evaluation, then apply what those tests reveal to make its agents more dependable and potentially less costly.