ChatGPT Shows Promise in Answering Patients' Questions to Urologists

The groundbreaking ChatGPT chatbot shows potential as a time-saving tool for responding to patient questions sent to the urologist's office, suggests a study in the September issue of Urology Practice®, an Official Journal of the American Urological Association (AUA). The journal is published in the Lippincott portfolio by Wolters Kluwer.

The artificial intelligence (AI) tool generated "acceptable" responses to nearly one-half of a sample of real-life patient questions, according to the new research by Michael Scott, MD, a urologist at Stanford University School of Medicine. "Generative AI technologies may play a valuable role in providing prompt, accurate responses to routine patient questions - potentially alleviating patients' concerns while freeing up clinic time and resources to address other complex tasks," Dr. Scott comments.

Can ChatGPT accurately answer questions from urology patients?

ChatGPT is an innovative large language model (LLM) that has sparked interest across a wide range of settings, including health and medicine. In some recent studies, ChatGPT has performed well in responding to various types of medical questions, although its performance in urology is less well-established.

Modern electronic health record (EHR) systems enable patients to send medical questions directly to their doctors. "This shift has been associated with an increased time burden of EHR use for physicians with a large portion of this attributed to patient in-basket messages," the researchers write. One study estimates that each message in a physician's inbox adds more than two minutes spent on the EHR.

Dr. Scott and colleagues collected 100 electronic patient messages requesting medical advice from a urologist at a men's health clinic. The messages were categorized by type of content and difficulty, then entered into ChatGPT. Five experienced urologists graded each AI-generated response in terms of accuracy, completeness, helpfulness, and intelligibility. Raters also indicated whether they would send each response to a patient.

Findings support 'generative AI technology to improve clinical efficiency'

The ChatGPT-generated responses were judged to be accurate, with an average score of 4.0 on a five-point scale; and intelligible, average score 4.7. Ratings of completeness and helpfulness were lower, but with little or no potential for harm. Scores were comparable for different types of question content (symptoms, postoperative concerns, etc).

"Overall, 47% of responses were deemed acceptable to send to patients," the researchers write. Questions rated as "easy" had a higher rate of acceptable responses: 56%, compared to 34% for "difficult" questions.

"These results show promise for the utilization of generative AI technology to help improve clinical efficiency," Dr. Scott and coauthors write. The findings "suggest the feasibility of integrating this new technology into clinical care to improve efficiency while maintaining quality of patient communication."

The researchers note some potential drawbacks of ChatGPT-generated responses to patient questions: "ChatGPT's model is trained on information from the Internet in general, as opposed to validated medical sources," with a "risk of generating inaccurate or misleading responses." The authors also highlight the need for safeguards to ensure patient privacy.

"While our study provides an interesting starting point, more research will be needed to validate the use of LLMs to respond to patient questions, in urology as well as other specialties," Dr. Scott comments. "This will be a potentially valuable healthcare application, particularly with continued advances in AI technology."

Scott M, Muncey W, Seranio N, Belladelli F, Del Giudice F, Li S, Ha A, Glover F, Zhang CA, Eisenberg ML.
Assessing Artificial Intelligence-Generated Responses to Urology Patient In-Basket Messages.
Urol Pract. 2024 Sep;11(5):793-798. doi: 10.1097/UPJ.0000000000000637

Most Popular Now

MEDICA 2024 + COMPAMED 2024: Adapted Hal…

11 - 14 November 2024, Düsseldorf, Germany. The final preparations for MEDICA 2024 and COMPAMED 2024 in Düsseldorf have begun. A total of more than 5,500 exhibitors from approximately 70 countries...

AI does Not Necessarily Lead to more Eff…

The use of artificial intelligence (AI) in hospitals and patient care is steadily increasing. Especially in specialist areas with a high proportion of imaging, such as radiology, AI has long...

Why the NHS is Seeking to Make Media Ser…

Opinion Article by Dean Moody, Healthcare Services Director, Airwave Healthcare. Tim Kelsey and Martha Lane Fox called for WiFi to be made available free of charge throughout the NHS back in...

Commission Joins Forces with Venture Cap…

The Commission has launched a Trusted Investors Network bringing together a group of investors ready to co-invest in innovative deep-tech companies in Europe together with the EU. The Union's investment...

An AI-Powered Pipeline for Personalized …

Ludwig Cancer Research scientists have developed a full, start-to-finish computational pipeline that integrates multiple molecular and genetic analyses of tumors and the specific molecular targets of T cells and harnesses...

Wearable Cameras Allow AI to Detect Medi…

A team of researchers says it has developed the first wearable camera system that, with the help of artificial intelligence (AI), detects potential errors in medication delivery. In a test whose...

Philips and Medtronic Advocacy Partnersh…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, and Medtronic Neurovascular, a leading innovator in neurovascular therapies, today announced a strategic advocacy partnership. Delivering timely stroke...

AI could Transform How Hospitals Produce…

A pilot study led by researchers at University of California San Diego School of Medicine found that advanced artificial intelligence (AI) could potentially lead to easier, faster and more efficient...

New AI Tool Predicts Protein-Protein Int…

Scientists from Cleveland Clinic and Cornell University have designed a publicly-available software and web database to break down barriers to identifying key protein-protein interactions to treat with medication. The computational tool...

Great Start for Ideas and Innovations: D…

8 - 10 April 2025, Berlin, Germany. From 15 October to 15 November 2024, the DMEA invites experts from business, science, politics and practice to actively participate in shaping the congress...

Start-Ups will Once Again Have a Starrin…

11 - 14 November 2024, Düsseldorf, Germany. The finalists in the 16th Healthcare Innovation World Cup and the 13th MEDICA START-UP COMPETITION have advanced from around 550 candidates based in 62...

G-Cloud 14 Makes it Easier for NHS to Bu…

NHS organisations will be able to save valuable time and resource in the procurement of technologies that can make a significant difference to patient experience, in the latest iteration of...