AI System Beats Doctors in Medical Simulations

An artificial intelligence system developed by Microsoft correctly diagnosed complex medical cases in a simulation more than twice as often as generalist physicians. This finding was detailed in a study published this week in the journal Science. The result marks a significant milestone in the ongoing effort to integrate advanced machine learning into clinical decision support systems across the region.

The diagnostic system, identified as MAI-DxO, functions by coordinating multiple artificial intelligence agents. These agents operate similarly to a panel of doctors by questioning patients, ordering necessary tests, and debating potential diagnoses before reaching a conclusion. In the specific simulation, which utilized difficult cases from the New England Journal of Medicine, the system achieved an accuracy rate of 83 percent. By comparison, primary care doctors and general internists who worked individually with reference materials were correct in 36 percent of the instances. It is important to note that cases of this complexity are typically handled by specialists or physician teams, who were not included in this specific comparison.

Dominic King, the vice president of Health at Microsoft AI, described the findings as substantial and exciting. He noted that the primary challenge in current healthcare delivery is the growing mismatch between the demand for and the supply of medical expertise. By publishing these results, Microsoft joins other major technology firms in asserting that their AI capabilities can match or exceed those of human doctors.

The study indicates that much of the performance improvement stems from more powerful underlying AI models, with Microsoft's orchestration system providing additional gains on top of that base capability. Microsoft tested the system with models from several companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest results were achieved using models from OpenAI, specifically GPT-5 and GPT-5.5.

Cost efficiency was also a key metric in the evaluation. When used alone with a basic prompt, GPT-5 scored 78.9 percent on complex cases at an estimated cost of 8,400 dollars per case. The orchestration system reduced that cost by approximately 3,000 dollars while raising accuracy by only about four points. On emergency department cases, there was little difference in either accuracy or cost between the standalone model and the assisted system.

With the newer GPT-5.5 model, released in April 2026, the results varied by case type. For complex cases, the system increased accuracy from 72 percent to nearly 87 percent at a slightly lower cost. However, these gains were not evident in emergency department cases, where both configurations scored about 85 percent at similar costs. The system proved most helpful on complex cases and when paired with less capable models, such as Gemini 2.5 Flash, which saw its score rise from 46.4 percent to nearly 69 percent with assistance.

Mustafa Suleyman, the CEO of Microsoft AI, stated that the company is collaborating with the Mayo Clinic to train a foundation model for healthcare, with a release planned for next year. He expressed confidence that this model will enable significant advancements. Suleyman believes that by 2027, the industry will be on the verge of a breakthrough in handling medical records, allowing for highly accurate predictions about how a patient's condition might unfold.

Suleyman, who previously co-founded DeepMind and joined Microsoft in 2024 to lead its AI unit, has long focused on using AI to solve medical issues. He expects healthcare models to improve significantly in capability within the next 24 to 36 months. He compared the current trajectory to previous progress in coding and chat, noting that rapid improvement occurs once a model is in production and being iterated upon with real-world usage.

Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate the clinical reasoning capabilities of the system across different hospital settings. Patients will receive standard treatment while those who consent will also engage with the AI. The company will then compare the AI's recommendations against those of human doctors. King described this as the biggest study of its kind, aiming to provide compelling evidence for future deployment at scale.

This study coincides with a broader manifesto from Microsoft on building medical superintelligence. The goal is to move beyond benchmarking individual tasks toward integrating with clinicians and healthcare systems for better overall outcomes. Suleyman stated that the company is committed to accelerating the introduction of this technology to the world, viewing it as a potential differentiator for Microsoft among other frontier labs.