Can ChatGPT be Counted On?

A study in the Journal of The National Cancer Institute Cancer Spectrum looked at chatbots and artificial intelligence (AI), as they become popular resources for cancer information. They found these resources give accurate information when asked about common cancer myths and misconceptions. In the first study of its kind, Skyler Johnson, MD, physician-scientist at Huntsman Cancer Institute and assistant professor in the department of radiation oncology at the University of Utah (the U), evaluated the reliability and accuracy of ChatGPT's cancer information.

Using the National Cancer Institute's (NCI) common myths and misconceptions about cancer, Johnson and his team found that 97% of the answers were correct. However, this finding comes with some important caveats, including a concern amongst the team that some of the ChatGPT answers could be interpreted incorrectly. "This could lead to some bad decisions by cancer patients. The team suggested caution when advising patients about whether they should use chatbots for information about cancer," says Johnson.

The study found reviewers were blinded, meaning they didn’t know whether the answers came from the chatbot or the NCI. Though the answers were accurate, reviewers found ChatGPT's language was indirect, vague, and in some cases, unclear.

"I recognize and understand how difficult it can feel for cancer patients and caregivers to access accurate information," says Johnson. "These sources need to be studied so that we can help cancer patients navigate the murky waters that exist in the online information environment as they try to seek answers about their diagnoses."

Incorrect information can harm cancer patients. In a previous study by Johnson and his team published in the Journal of the National Cancer Institute, they found that misinformation was common on social media and had the potential to harm cancer patients.

The next steps are to evaluate how often patients are using chatbots to seek out information about cancer, what questions they are asking, and whether AI chatbots provide accurate answers to uncommon or unusual questions about cancer.

The study was supported by the National Institutes of Health/National Cancer Institute including P30 CA042014 and Huntsman Cancer Foundation.

Johnson SB, King AJ, Warner EL, Aneja S, Kann BH, Bylund CL.
Using ChatGPT to evaluate cancer myths and misconceptions: artificial intelligence and cancer information.
JNCI Cancer Spectr. 2023 Mar 1;7(2):pkad015. doi: 10.1093/jncics/pkad015

Most Popular Now

AI Identifies Anti-Aging Drug Candidates…

A new publication in the May issue of Nature Aging by researchers from Integrated Biosciences, a biotechnology company combining synthetic biology and machine learning to target aging, demonstrates the power...

AI could Improve Heart Attack Diagnosis …

An algorithm developed using artificial intelligence (AI) could soon be used by doctors to diagnose heart attacks with better speed and accuracy than ever before, according to new research from...

AI Predicts Future Pancreatic Cancer

An artificial intelligence (AI) tool has successfully identified people at the highest risk for pancreatic cancer up to three years before diagnosis using solely the patients’ medical records, according to...

New Algorithm can Predict Diabetic Kidne…

Researchers from Sanford Burnham Prebys and the Chinese University of Hong Kong have developed a computational approach to predict whether a person with type 2 diabetes will develop kidney disease...

AI Voice Coach Shows Promise in Depressi…

Artificial intelligence (AI) could be a useful tool in mental health treatment, according to the results of a new pilot study led by University of Illinois Chicago researchers. The study...

AI could Run a Million Microbial Experim…

An artificial intelligence (AI) system enables robots to conduct autonomous scientific experiments - as many as 10,000 per day - potentially driving a drastic leap forward in the pace of...

Scientists develop AI tool to predict Pa…

Scientists from UNSW Sydney with collaborators at Boston University have developed a tool that shows early promise in detecting Parkinson’s disease years before the first symptoms start appearing. In research published...

Better than Humans: AI in Intensive Care…

In the future, artificial intelligence (AI) will play an important role in medicine. In diagnostics, successful tests have already been performed: for example, the computer can learn to categorise images...

Could Online Gaming Social Networks Have…

For millions of Americans playing some type of video game is a daily occurrence. Games can be a welcome form of entertainment and relaxation for many, and the internet can...

ChatGPT Passes Radiology Board Exam

The latest version of ChatGPT passed a radiology board-style exam, highlighting the potential of large language models but also revealing limitations that hinder reliability, according to two new research studies...

Siemens Healthineers Opens State-of-the-…

Siemens Healthineers has opened its new Education & Development Center (EDC) in Erlangen. The open-plan building offers space for the currently 240 trainees and integrated degree program participants in Erlangen...

Siemens Healthineers Invests 80 Million …

Siemens Healthineers is building a new factory in Forchheim for the cultivation of crystals for semiconductor production. The total investment amounts to 80 million euros. The new factory is expected...