Microsoft is presenting a new AI system as a major advance in medical diagnosis. In an experiment built around complex case studies, the company says its tool diagnosed ailments four times more accurately than a panel of human physicians while also choosing less expensive tests and procedures.
The results are striking, but the same source also makes clear that this is not yet a proven replacement for clinical judgment. The work points to where medical AI may be headed, while raising familiar questions about validation, cost, bias, and how closely a benchmark can reflect real care.
What Microsoft tested
The project centers on a system called the MAI Diagnostic Orchestrator, or MAI-DxO. Microsoft’s researchers designed it to work less like a single chatbot and more like a group of expert systems debating possible answers.
To build the test, the team used 304 case studies from the New England Journal of Medicine. A language model converted each case into a step-by-step process resembling the work a doctor might do while moving from symptoms to tests, analysis, and diagnosis.
That test was called the Sequential Diagnosis Benchmark. Its purpose was not simply to ask whether an AI model could name a disease after reading a complete medical file. Instead, it tried to imitate the sequence of decisions involved in diagnosis: consider the information available, decide what to check next, interpret the result, and keep narrowing the answer.
MAI-DxO then queried several leading AI models, including OpenAI’s GPT, Google’s Gemini, Anthropic’s Claude, Meta’s Llama, and xAI’s Grok. Microsoft says this orchestration approach loosely mirrors the way multiple human experts might work together.
The headline result
In Microsoft’s experiment, MAI-DxO reached an accuracy of 80 percent. The doctors in the same study reached 20 percent.
Microsoft also says the system reduced costs by 20 percent by choosing less expensive tests and procedures. That cost result matters because health care spending is a major issue, particularly in the US, and diagnostic pathways often involve choices about which tests are worth ordering.
Mustafa Suleyman, CEO of Microsoft’s artificial intelligence arm, described the work as “a genuine step toward medical superintelligence.” He also pointed to the system’s multi-agent design, saying, “This orchestration mechanism—multiple agents that work together in this chain-of-debate style—that's what's going to drive us closer to medical superintelligence.”
Dominic King, a vice president at Microsoft involved with the project, framed the result in practical terms: “Our model performs incredibly well, both getting to the diagnosis and getting to that diagnosis very cost effectively,” he said.
Why this differs from earlier medical AI work
AI is already used in parts of the US health care industry, including radiology workflows that help interpret scans. More recent multimodal AI models have raised the possibility of broader diagnostic tools that can reason across different kinds of medical information.
The Microsoft project sits within a growing body of research showing that large language models can diagnose disease when given access to medical records. Both Microsoft and Google have published papers in the last few years on that broader question.
The distinction here is the workflow. Microsoft’s new research aims to reflect more of the process physicians use: reviewing symptoms, ordering tests, and updating the analysis until a diagnosis emerges. That makes the benchmark more ambitious than a simple one-shot question-and-answer test.
Microsoft has described its combination of several frontier AI models as “a path to medical superintelligence” in a blog post about the project. The phrase is bold, but the source also shows why the company is using it: the system is not just producing an answer, it is coordinating multiple models through a diagnostic process.
The cautions experts raised
Outside experts treated the work as important, but not as a final verdict. David Sontag, a scientist at MIT and cofounder of Layer Health, said the project is valuable because it more closely reflects how physicians operate and because it addresses methodological concerns. “That’s what makes this paper strong,” he said.
But Sontag also warned that the comparison with doctors has limits. In the study, doctors were asked not to use additional tools to help with diagnosis. That may not match how doctors work in real practice.
He also said it remains uncertain whether the AI system would meaningfully reduce costs outside the benchmark. Real medical decisions can include factors that the AI may not capture, including a patient’s tolerance for a procedure or whether a particular medical instrument is available.
Eric Topol, a scientist at the Scripps Research Institute, also saw significance in the work. “This is an impressive report because it tackles highly complex cases for diagnosis,” he said. He added that showing AI could theoretically reduce the cost of medical care is novel.
The source also notes a broader concern for AI in health care: bias from training data that is skewed toward particular demographics. That issue is not specific to MAI-DxO alone, but it remains central to any discussion of diagnostic AI tools.
What has to happen next
Microsoft has not decided whether it will commercialize the technology. An executive who spoke on the condition of anonymity said the company could integrate it into Bing to help users diagnose ailments, or develop tools that help medical experts improve or even automate patient care.
Suleyman said, “What you'll see over the next couple of years is us doing more and more work proving these systems out in the real world.” That real-world proof is the key gap between an impressive benchmark and a dependable medical product.
Both Topol and Sontag said the next step before broad deployment would be a clinical trial comparing the tool’s performance with real doctors treating real patients. Sontag said that would also allow a more rigorous evaluation of cost.
For now, Microsoft’s medical AI benchmark is best read as a serious signal, not a finished answer. MAI-DxO performed strongly in a structured diagnostic test, but the harder question is whether that performance holds when medicine becomes less controlled, more personal, and tied to real patient choices.