ChatGPT Shows Human-Level Assessment of Brain Tumor MRI Reports

As artificial intelligence advances, its uses and capabilities in real-world applications continue to reach new heights that may even surpass human expertise. In the field of radiology, where a correct diagnosis is crucial to ensure proper patient care, large language models, such as ChatGPT, could improve accuracy or at least offer a good second opinion.

To test its potential, graduate student Yasuhito Mitsuyama and Associate Professor Daiju Ueda's team at Osaka Metropolitan University’s Graduate School of Medicine led the researchers in comparing the diagnostic performance of GPT-4 based ChatGPT and radiologists on 150 preoperative brain tumor MRI reports. Based on these daily clinical notes written in Japanese, ChatGPT, two board-certified neuroradiologists, and three general radiologists were asked to provide differential diagnoses and a final diagnosis.

Subsequently, their accuracy was calculated based on the actual diagnosis of the tumor after its removal. The results stood at 73% for ChatGPT, a 72% average for neuroradiologists, and 68% average for general radiologists. Additionally, ChatGPT’s final diagnosis accuracy varied depending on whether the clinical report was written by a neuroradiologist or a general radiologist. The accuracy with neuroradiologist reports was 80%, compared to 60% when using general radiologist reports.

"These results suggest that ChatGPT can be useful for preoperative MRI diagnosis of brain tumors," stated graduate student Mitsuyama. "In the future, we intend to study large language models in other diagnostic imaging fields with the aims of reducing the burden on physicians, improving diagnostic accuracy, and using AI to support educational environments."

Mitsuyama Y, Tatekawa H, Takita H, Sasaki F, Tashiro A, Oue S, Walston SL, Nonomiya Y, Shintani A, Miki Y, Ueda D.
Comparative analysis of GPT-4-based ChatGPT's diagnostic performance with radiologists using real-world radiology reports of brain tumors.
Eur Radiol. 2024 Aug 28. doi: 10.1007/s00330-024-11032-8

Most Popular Now

Adults don't Trust Health Care to U…

A study finds that 65.8% of adults surveyed had low trust in their health care system to use artificial intelligence responsibly and 57.7% had low trust in their health care...

The 10 Year Health Plan: What do We Need…

Opinion Article by Piyush Mahapatra, Consultant Orthopaedic Surgeon and Chief Innovation Officer at Open Medical. There is a new ten-year plan for the NHS. It will "focus efforts on preventing, as...

People's Trust in AI Systems to Mak…

Psychologists warn that AI's perceived lack of human experience and genuine understanding may limit its acceptance to make higher-stakes moral decisions. Artificial moral advisors (AMAs) are systems based on artificial...

AI Model can Read ECGs to Identify Femal…

A new AI model can flag female patients who are at higher risk of heart disease based on an electrocardiogram (ECG). The researchers say the algorithm, designed specifically for female patients...

New AI Tool Mimics Radiologist Gaze to R…

Artificial intelligence (AI) can scan a chest X-ray and diagnose if an abnormality is fluid in the lungs, an enlarged heart or cancer. But being right is not enough, said...

Relationship Between Sleep and Nutrition…

Diet and sleep, which are essential for human survival, are interrelated. However, recently, various services and mobile applications have been introduced for the self-management of health, allowing users to record...

To be Happier, Take a Vacation... from Y…

Today, nearly every American - 91% - owns a cellphone that can access the internet, according to the Pew Research Center. In 2011, only about one-third did. Another study finds...

DMEA 2025 - Innovations, Insights and Ne…

8 - 10 April 2025, Berlin, Germany. Less than 50 days to go before DMEA 2025 opens its doors: Europe's leading event for digital health will once again bring together experts...

Researchers Find Telemedicine may Help R…

Low-value care - medical tests and procedures that provide little to no benefit to patients - contributes to excess medical spending and both direct and cascading harms to patients. A...

AI Revolutionizes Glaucoma Care

Imagine walking into a supermarket, train station, or shopping mall and having your eyes screened for glaucoma within seconds - no appointment needed. With the AI-based Glaucoma Screening (AI-GS) network...

North Cumbria Integrated Care Signs 10-Y…

North Cumbria Integrated Care NHS Foundation Trust (NCIC) has signed a long-term agreement for use of the Alcidion Miya Precision platform, to provide an electronic patient record (EPR) for the...

AI Accelerates Discovery of Neurodevelop…

Researchers have developed an artificial intelligence (AI) approach that accelerates the identification of genes that contribute to neurodevelopmental conditions such as autism spectrum disorder, epilepsy and developmental delay. This new...