In a paper published on 22nd October in Science, a team of Microsoft researchers claimed their AI system could solve complicated medical cases more accurately than general physicians, with accuracy over twice as high, and the system could diagnose patients step-by-step like doctors do.

It uses an “orchestrator” to identify and synthesise the questions to ask a patient and the exams and tests to order based on the task, which is done by simulating a panel of clinical experts working together to solve diagnostic problems, the report said.

Current medical AI models rely on a one-shot paradigm which treats a patient as a whole, and uses all of their medical data (sx, lab, imaging) to make a diagnosis.

With the orchestrator coordinating the steps to reach a diagnosis, ​the AI model first decides what to ask about the patient’s history and what is significant ​in the physical exam. It is based on what it learns that it decides on what tests to order, takes the test results into account and orders further tests accordingly.

All the while, it is updating its working hypotheses and plans, the research leaders said ​in an interview.

The model with the diagnostic orchestrator achieved over twice the accuracy in “working up” the patient and in making the diagnosis compared to general physicians relying on textbooks and the internet, for more than 300 highly unusual and confusing medical dilemmas.

Mechanisms were also better than human doctors at tackling scenarios culled from a large emergency room admission database.

These sorts of older systems are typically evaluated using multiple-choice questions, researchers said.

“AI is now very good at achieving a 100% pass rate on those,” Dr Dominick King said, who is the corporate vice president of the health unit at Microsoft AI, which is working on developing the new technology.

Of course, that isn’t real-world, but no patient ever came to see me with a nice summary,

The researchers further discovered that the cost of diagnostic processes is less when the orchestrator plans these processes than when they are performed by the human doctor.

“We’re not just measuring the accuracy of the systems; we’re also measuring how judiciously and carefully it goes through a diagnostic work-up,” said Harsha Nori, head of AI at Microsoft AI Health.

The researchers will then work with Beth Israel Deaconess Medical Centre in Boston to evaluate the tool’s effectiveness in triaging people in the waiting rooms of their emergency department.