ChatGPT Extracts Data for Ischaemic Stroke Almost Perfectly

In an ischaemic stroke, an artery in the brain is blocked by blood clots and the brain cells can no longer be supplied with blood as a result. Doctors must therefore act quickly and unblock the artery with the help of catheters. During the so-called mechanical thrombectomy, a lot of data has to be recorded and then transferred to various registers. Dr Nils Lehnen, senior physician at the Clinic for Diagnostic and Interventional Neuroradiology and Paediatric Neuroradiology at the University Hospital Bonn (UKB), has now discovered in a study that ChatGPT could be a great help in this data transfer. The results have now been published in the specialist journal "RADIOLOGY".

When did the patient arrive, when was a CT scan performed, when was the first puncture, when could the blood flow be restored,... During mechanical thrombectomy, a range of data must be recorded in the patient report and then manually transferred to various registers for the clinical outcome and for prospective studies. "This is a labour-intensive task that is also prone to transcription errors," says Dr Nils Lehnen, who also conducts research at the University of Bonn. "We therefore asked ourselves whether an AI such as ChatGPT could perform this transfer faster and possibly even more reliably."

In radiology, ChatGPT is already being tested in various procedures - for example, in the simplification of reports or in answering patient questionson breast cancer screening. However, whether ChatGPT can correctly extract data from free-text reports of a mechanical thrombectomy for a database and simultaneously generate clinical data was previously unexplored and was the research objective of this new study.

Dr Lehnen's research group first created a German prompt for ChatGPT and tested it on 20 reports in order to identify errors and subsequently adapt the prompt. After the correction, the data extraction using ChatGPT was tested on 100 internal reports from the UKB. For optimal comparison, an experienced neuroradiologist also compiled the results without seeing the ChatGPT evaluation. The researchers then compared the results and found that ChatGPT had correctly extracted 94 per cent of data entries and no post-processing was required. The researchers only considered the ChatGPT data entries that exactly matched that of the expert to be correct. Any deviations, such as additional symbols, punctuation marks or synonyms, were categorised as incorrect.

To validate these results, the researchers tested a further 30 external reports with the same prompt. ChatGPT achieved 90 per cent correct data entries.

"This suggests that ChatGPT could be an alternative to manually retrieving this data," says Dr Lehnen. "However, the reports and the prompt were only created by us in German, so the results of our study may need to be confirmed for other languages. In addition, we still observed poor results for certain data points, which shows that human supervision is still needed. However, we expect that further optimisation of the prompt will further improve the results and that ChatGPT can make work easier in this area in the future."

Lehnen NC, Dorn F, Wiest IC, Zimmermann H, Radbruch A, Kather JN, Paech D.
Data Extraction from Free-Text Reports on Mechanical Thrombectomy in Acute Ischemic Stroke Using ChatGPT: A Retrospective Analysis.
Radiology. 2024 Apr;311(1):e232741. doi: 10.1148/radiol.232741

Most Popular Now

Herefordshire and Worcestershire Health …

Herefordshire and Worcestershire Health and Care NHS Trust has successfully implemented Alcidion's Miya Precision platform to streamline bed management workflow across seven community hospitals in Worcestershire. The trust delivers community...

With Huge Patient Dataset, AI Accurately…

Scientists have designed a new artificial intelligence (AI) model that emulates randomized clinical trials at determining the treatment options most effective at preventing stroke in people with heart disease. The model...

A Shortcut for Drug Discovery

For most human proteins, there are no small molecules known to bind them chemically (so called "ligands"). Ligands frequently represent important starting points for drug development but this knowledge gap...

New Horizon Europe Funding Boosts Europe…

The European Commission has announced the launch of new Horizon Europe calls, with a substantial funding pool of over €112 million. These calls are aimed primarily at pioneering projects in...

Cleveland Clinic Study Finds AI can Deve…

Cleveland Clinic researchers developed an artficial intelligence (AI) model that can determine the best combination and timeline to use when prescribing drugs to treat a bacterial infection, based solely on...

New AI-Technology Estimates Brain Age Us…

As people age, their brains do, too. But if a brain ages prematurely, there is potential for age-related diseases such as mild-cognitive impairment, dementia, or Parkinson's disease. If "brain age...

Radboud University Medical Center and Ph…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, and Radboud University Medical Center have signed a hospital-wide, long-term strategic partnership that delivers the latest patient monitoring...

GPT-4, Google Gemini Fall Short in Breas…

Use of publicly available large language models (LLMs) resulted in changes in breast imaging reports classification that could have a negative effect on patient management, according to a new international...

ChatGPT fails at heart risk assessment

Despite ChatGPT's reported ability to pass medical exams, new research indicates it would be unwise to rely on it for some health assessments, such as whether a patient with chest...

Study Shows ChatGPT Failed when Challeng…

With artificial intelligence (AI) poised to become a fundamental part of clinical research and decision making, many still question the accuracy of ChatGPT, a sophisticated AI language model, to support...

Virtual Reality Shows Promise in Fightin…

A new study published in JMIR Mental Health sheds light on the promising role of virtual reality (VR) in treating major depressive disorder (MDD). Titled "Examining the Efficacy of Extended...

AXREM and Highland Marketing Partner to …

AXREM represents member companies that collectively provide UK hospitals with most of their diagnostic medical imaging technology, and radiotherapy equipment. The association has seen substantial growth in recent years, with membership...