Microsoft's AI Beat Doctors 83% to 36%. Here's What That Actually Means

October 26, 2026 — Microsoft published a study in Science last week that has been making headlines around the world. Its AI system, called MAI-DxO, correctly diagnosed 83% of complex medical cases in a simulation. The human doctors it was compared against scored 36%.

That is more than double. And when the doctors were stripped of reference materials, their accuracy dropped to 22% — making the AI nearly four times as accurate [citation:1][citation:9].

The numbers are striking. But before you conclude that AI is about to replace your doctor, there is one detail in the study that changes how you should read the headline.

How MAI-DxO Works: A Panel of AI Doctors

What makes MAI-DxO different from asking ChatGPT a medical question is the architecture. Instead of a single model receiving a full case summary and spitting out an answer, MAI-DxO coordinates multiple AI agents that each play a different role [citation:2][citation:6].

One agent proposes possible diagnoses. Another decides which tests to order. A third challenges the assumptions being made. A fourth weighs the cost of each next step. They work sequentially: start with the chief complaint, ask follow-up questions one at a time, order tests only when necessary, and revise the diagnosis as new information comes in [citation:5][citation:7].

The system is also model-agnostic. Microsoft tested it with GPT-5, GPT-5.5, Gemini, Claude, and DeepSeek. The best results came with OpenAI's GPT-5 and GPT-5.5. The study's own data shows that much of the performance gain comes from the underlying model itself — DxO's orchestration adds a few percentage points on top, not a revolution [citation:2][citation:4].

The Caveat That Changes Everything

Here is the part of the study that the headlines tend to skip.

The complex cases used in the benchmark came from the New England Journal of Medicine. These are rare, difficult, and unusual presentations. In a real hospital, a case of this complexity would not be handed to a general practitioner working alone with a textbook. It would be escalated to a specialist or a multi-disciplinary team [citation:1][citation:6][citation:7].

The comparison group in Microsoft's study was generalist physicians and internists. Specialists were not included. So the 83% vs. 36% gap is real, but it is not a fair comparison to the standard of care that these cases would actually receive in a clinical setting.

There is a second limitation: these were simulated cases. The AI and the doctors worked with text descriptions, not real patients. No physical examinations. No medical imaging. No patients who change their story halfway through, or who have three other conditions that complicate the picture. Microsoft is now starting a prospective clinical study at Beth Israel Deaconess Medical Center in Boston to test the system against real doctors and real patients [citation:1][citation:6]. Until those results are in, the 83% figure is a benchmark score, not a clinical outcome.

What This Means for You

For most people, this study does not mean that an AI will diagnose you anytime soon. But it does point toward something more plausible in the near term: AI as a diagnostic assistant.

Microsoft AI CEO Mustafa Suleyman has called this a step toward "medical superintelligence" — a system that surpasses the strongest human-led health systems across the full arc of care [citation:4][citation:6]. That is a long-term vision. The near-term reality is more modest: an AI that can reason through a case step by step, ask the right questions, and flag possibilities a tired doctor might miss. If the clinical trials at Beth Israel confirm that it works with real patients, that would already be a meaningful contribution. It would not replace doctors. It would give them a better safety net.

Key Takeaways