Can ChatGPT be Counted On?

A study in the Journal of The National Cancer Institute Cancer Spectrum looked at chatbots and artificial intelligence (AI), as they become popular resources for cancer information. They found these resources give accurate information when asked about common cancer myths and misconceptions. In the first study of its kind, Skyler Johnson, MD, physician-scientist at Huntsman Cancer Institute and assistant professor in the department of radiation oncology at the University of Utah (the U), evaluated the reliability and accuracy of ChatGPT's cancer information.

Using the National Cancer Institute's (NCI) common myths and misconceptions about cancer, Johnson and his team found that 97% of the answers were correct. However, this finding comes with some important caveats, including a concern amongst the team that some of the ChatGPT answers could be interpreted incorrectly. "This could lead to some bad decisions by cancer patients. The team suggested caution when advising patients about whether they should use chatbots for information about cancer," says Johnson.

The study found reviewers were blinded, meaning they didn’t know whether the answers came from the chatbot or the NCI. Though the answers were accurate, reviewers found ChatGPT's language was indirect, vague, and in some cases, unclear.

"I recognize and understand how difficult it can feel for cancer patients and caregivers to access accurate information," says Johnson. "These sources need to be studied so that we can help cancer patients navigate the murky waters that exist in the online information environment as they try to seek answers about their diagnoses."

Incorrect information can harm cancer patients. In a previous study by Johnson and his team published in the Journal of the National Cancer Institute, they found that misinformation was common on social media and had the potential to harm cancer patients.

The next steps are to evaluate how often patients are using chatbots to seek out information about cancer, what questions they are asking, and whether AI chatbots provide accurate answers to uncommon or unusual questions about cancer.

The study was supported by the National Institutes of Health/National Cancer Institute including P30 CA042014 and Huntsman Cancer Foundation.

Johnson SB, King AJ, Warner EL, Aneja S, Kann BH, Bylund CL.
Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information.
JNCI Cancer Spectr. 2023 Mar 1;7(2):pkad015. doi: 10.1093/jncics/pkad015

Most Popular Now

AI can Help Improve Emergency Room Admis…

Generative artificial intelligence (AI), such as GPT-4, can help predict whether an emergency room patient needs to be admitted to the hospital even with only minimal training on a limited...

Philips ePatch and AI Analytics Platform…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, announced the successful nationwide rollout of its ambulatory cardiac monitoring service in Spain using its unique wearable ePatch...

Comprehensive Bibliographic Dataset Adva…

A groundbreaking study published in Health Data Science, a Science Partner Journal, introduces a curated bibliographic dataset that aims to revolutionize the landscape of Health Artificial Intelligence (AI) research. Led...

AI Health Coach Lowers Blood Pressure an…

A new study in JMIR Cardio, published by JMIR Publications, shows that a fully digital, artificial intelligence (AI)-driven lifestyle coaching program can effectively reduce blood pressure (BP) in adults with...

Will Generative AI Change the Way Univer…

Since the launch of ChatGPT 3 in November 2022, we've been abuzz with talk of artificial intelligence: is it an unprecedented opportunity, or will it rob everyone of jobs and...

New Deep Learning Model is 'Game Ch…

Research led by the University of Plymouth has shown that a new deep learning AI model can identify what happens and when during embryonic development, from video. Published in the Journal...

Huge NHS Cloud Deals Mean Tough Question…

Opinion Article by Chris Scarisbrick, Deputy Managing Director, Sectra. The largest public cloud projects to ever take place within the NHS are beginning. Regional procurements for public cloud hosted diagnostic imaging...

AI Tech should Augment Physician Decisio…

The use of artificial intelligence (AI) in clinical health care has the potential to transform health care delivery but it should not replace physician decision-making, says the American College of...

A Three-Point Plan for Digital Delivery

Sam Shah has seen health tech policy up-close and worries that little progress has been made over the past five-years. However, he has a plan for any health and social...

Facial Thermal Imaging + AI Accurately P…

A combination of facial thermal imaging and artificial intelligence (AI) can accurately predict the presence of coronary artery disease, finds research published in the open access journal BMJ Health &...

New AI Algorithm Detects Rare Epileptic …

More than 3.4 million people in the US and 65 million people worldwide have epilepsy, a neurological disorder that affects the nervous system and causes seizures. One in 26 people...

Siemens Healthineers Debuts New Cardiolo…

Siemens Healthineers announces new cardiology applications with artificial intelligence for the Acuson Sequoia ultrasound system, as well as a new 4D transesophageal (TEE) transducer for cardiology exams. These cardiology applications...