ChatGPT Performs as Well as Doctors for Suggesting the Most Likely Diagnoses in the Emergency Medicine Department

The artificial intelligence chatbot ChatGPT performed as well as a trained doctor in suggesting likely diagnoses for patients being assessed in emergency medicine departments, in a pilot study to be presented at the European Emergency Medicine Congress.

Researchers say a lot more work is needed, but their findings suggest the technology could one day support doctors working in emergency medicine, potentially leading to shorter waiting times for patients.

The study was by Dr Hidde ten Berg, from the department of emergency medicine and Dr Steef Kurstjens, from the department of clinical chemistry and haematology, both at Jeroen Bosch Hospital, 's-Hertogenbosch, The Netherlands.

Dr ten Berg told the Congress: "Like a lot of people, we have been trying out ChatGPT and we were intrigued to see how well it worked for examining some complex diagnostic cases. So, we set up a study to assess how well the chatbot worked compared to doctors with a collection of emergency medicine cases from daily practice."

The research, which is also published this month in the Annals of Emergency Medicine, included anonymised details on 30 patients who were treated at Jeroen Bosch Hospital’s emergency department in 2022. The researchers entered physicians’ notes on patients’ signs, symptoms and physical examinations into two versions of ChatGPT (the free 3.5 version and the subscriber 4.0 version). They also provided the chatbot with results of lab tests, such as blood and urine analysis. For each case, they compared the shortlist of likely diagnoses generated by the chatbot to the shortlist made by emergency medicine doctors and to the patient’s correct diagnosis.

They found a large overlap (around 60%) between the shortlists generated by ChatGPT and the doctors. Doctors had the correct diagnosis within their top five likely diagnoses in 87% of the cases, compared to 97% for ChatGPT version 3.5 and 87% for version 4.0.

Dr ten Berg said: "We found that ChatGPT performed well in generating a list of likely diagnoses and suggesting the most likely option. We also found a lot of overlap with the doctors' lists of likely diagnoses. Simply put, this indicates that ChatGPT was able suggest medical diagnoses much like a human doctor would.

"For example, we included a case of a patient presenting with joint pain that was alleviated with painkillers, but redness, joint pain and swelling always recurred. In the previous days, the patient had a fever and sore throat. A few times there was a discolouration of the fingertips. Based on the physical exam and additional tests, the doctors thought the most likely diagnosis was probably rheumatic fever, but ChatGPT was correct with its most likely diagnosis of vasculitis.

"It's vital to remember that ChatGPT is not a medical device and there are concerns over privacy when using ChatGPT with medical data. However, there is potential here for saving time and reducing waiting times in the emergency department. The benefit of using artificial intelligence could be in supporting doctors with less experience, or it could help in spotting rare diseases."

Professor Youri Yordanov from the St Antoine Hospital emergency department (APHP Paris), France, is Chair of the EUSEM 2023 abstract committee and was not involved in the research. He said: "We are a long way from using ChatGPT in the clinic, but it’s vital that we explore new technology and consider how it could be used to help doctors and their patients. People who need to go to the emergency department want to be seen as quickly as possible and to have their problem correctly diagnosed and treated. I look forward to more research in this area and hope that it might ultimately support the work of busy health professionals."

Berg HT, van Bakel B, van de Wouw L, Jie KE, Schipper A, Jansen H, O'Connor RD, van Ginneken B, Kurstjens S.
ChatGPT and Generating a Differential Diagnosis Early in an Emergency Department Presentation.
Ann Emerg Med. 2023 Sep 9:S0196-0644(23)00642-X. doi: 10.1016/j.annemergmed.2023.08.003

Most Popular Now

AI Predictions for Colorectal Cancer: On…

Colorectal cancer (CRC) ranks second in leading causes of cancer-related deaths globally, according to the WHO. For the first time, researchers from Helmholtz Munich and the University of Technology Dresden...

Combining AI Models Improves Breast Canc…

Combining artificial intelligence (AI) systems for short- and long-term breast cancer risk results in an improved cancer risk assessment, according to a study published in Radiology, a journal of the...

ChatGPT Shows 'Impressive' Acc…

A new study led by investigators from Mass General Brigham has found that ChatGPT was about 72 percent accurate in overall clinical decision making, from coming up with possible diagnoses...

Healthcare Chatbot: Expand Support with …

The Danish eHealth platform, sundhed.dk, has faced a substantial surge in requests from Danish citizens and has swiftly expanded its support and effectively adapt to the ongoing changes in queries due...

WiFi SPARK's Healthcare Business Re…

Leading WiFi provider WiFi SPARK is rebranding its healthcare arm as SPARK Technology Services Limited. The new identity marks the completion of the integration of the former Hospedia bedside unit...

ChatGPT is Debunking Myths on Social Med…

ChatGPT could help to increase vaccine uptake by debunking myths around jab safety, say the authors of a study published in the peer-reviewed journal Human Vaccines and Immunotherapeutics. The researchers asked...

Online AI-Based Test for Parkinson'…

An artificial intelligence (AI) tool developed by researchers at the University of Rochester can help people with Parkinson's disease remotely assess the severity of their symptoms within minutes. A study...

AI Performs Comparably to Human Readers …

Using a standardized assessment, researchers in the UK compared the performance of a commercially available artificial intelligence (AI) algorithm with human readers of screening mammograms. Results of their findings were...

Siemens Healthineers Expands Production …

Siemens Healthineers is expanding its site in Rudolstadt, Germany. By mid 2024, a new manufacturing building will be built on the site. The new manufacturing plant will produce electron accelerators...

More Cases of Breast Cancer Detected wit…

One radiologist supported by AI detected more cases of breast cancer in screening mammography than two radiologists working together, reports the ScreenTrustCAD study from Karolinska Institutet in The Lancet Digital...

MEDICA 2023 + COMPAMED 2023: "Where…

13 - 16 November 2023, Düsseldorf, Germany. The medical technology market is in worldwide motion and the signs ahead of MEDICA 2023 and COMPAMED 2023 in Düsseldorf as the internationally leading...

Smartphone Technology Expected to Advanc…

Since the 1980s, we have known that neurological soft signs (NSS) can distinguish people with schizophrenia from psychiatrically healthy individuals. NSS are subtle neurological impairments that principally manifest as decreased...