AI Diagnoses Rare Diseases Better Than Doctors, Major Study Finds

An artificial intelligence system has outperformed human doctors on a range of demanding clinical tasks, including diagnosing patients and managing their care in emergency settings, according to a new study led by Harvard researchers and published in the journal Science. The AI model tested, OpenAI's "o1 preview," was able to carry out triage in emergency rooms, recommend diagnostic tests, and handle case management at a level that matched or surpassed even highly experienced physicians.

The researchers tested the model on 76 real emergency room cases at a Boston hospital, assessing it at three points: initial triage, first contact with a doctor, and admission to a ward or intensive care unit. Two independent doctors, who did not know whether the assessments came from the AI or from human specialists, judged the AI to have matched or beaten human performance at every stage. It proved particularly strong during initial triage, when the least information was available.

The AI also excelled at diagnosing rare and complex conditions, performing well on famously difficult cases from Massachusetts General Hospital that have long been used to benchmark computer diagnostics going back to 1959. On tasks involving clinical management, such as recommending antibiotics or navigating end-of-life conversations, the AI significantly outperformed both earlier AI models and humans using standard tools such as Google Search.

Despite the striking results, the study's authors were careful to stress that the findings do not mean AI should replace doctors. Senior co-author Arjun Manrai noted that the study relied entirely on written text, whereas real clinical practice involves listening to patients, reviewing X-rays, interpreting ECGs and processing many other forms of data. He described the results as evidence of "a profound change in technology that will reshape medicine," but argued that what is needed now are rigorous clinical trials to understand how best to deploy it.

Co-author Adam Rodman suggested two areas where AI could prove especially useful: supporting triage by scanning electronic health records for missed diagnoses, and providing a reliable second opinion. A 2025 Elsevier study found that one in five clinicians was already consulting an AI model for a second opinion. Both researchers stressed that AI should work alongside doctors rather than replace them, and that human contact remains central to medicine.