Google & ChatGPT Have Mixed Results in Medical Information Queries

When you need accurate information about a serious illness, should you go to Google or ChatGPT?

An interdisciplinary study led by University of California, Riverside, computer scientists found that both internet information gathering services have strengths and weaknesses for people seeking information about Alzheimer's disease and other forms of dementia. The team included clinical scientists from the University of Alabama and Florida International University.

Google provides the most current information, but query results are skewed by service and product providers seeking customers, the researchers found. ChatGPT, meanwhile, provides more objective information, but it can be outdated and lacks the sources of its information in its narrative responses.

"If you pick the best features of both, you can build a better system, and I think that this is what will happen in the next couple of years," said Vagelis Hristidis, a professor of computer science and engineering in UCR's Bourns College of Engineering.

In their study, Hristidis and his co-authors submitted 60 queries to both Google and ChatGPT that would be typical submissions from people living with dementia and their families.

The researchers focused on dementia because more than 6 million Americans are impacted by Alzheimer's disease or a related condition, said study co-author Nicole Ruggiano, a professor of social work at the University of Alabama.

"Research also shows that caregivers of people living with dementia are among the most engaged stakeholders in pursuing health information, since they often are tasked with making decisions for their loved one's care," Ruggiano said.

Half of the queries submitted by the researchers sought information about the disease processes, while the other half sought information on services that could assist patients and their families.

The results were mixed.

"Google has more up-to-date information, and covers everything,” Hristidis said. “Whereas ChatGPT is trained every few months. So, it is behind. Let's say there's some new medicine that just came out last week, you will not find it on ChatGPT."

While dated, ChatGPT provided more reliable and accurate information than Google. This is because the ChatGPT creators at OpenAI choose the most reliable websites when they train ChatGPT through computationally intensive machine learning. Yet, users are left in dark about specific sources of information because the resulting narratives are void of references.

Google, however, has a reliability problem because it essentially "covers everything from the reliable sources to advertisements," Hristidis said.

In fact, advertisers pay Google for their website links to appear at the top of search result pages. So, users often first see links to websites of for-profit companies trying to sell them care-related services and products. Finding reliable information from Google searches thus requires a level of user skill and experience, Hristidis said.

Co-author Ellen Brown, an associate professor of nursing at the Florida International University, pointed out that families need timely information about Alzheimer's.

"Although there is no cure for the disease, many clinical trials are underway and recently a promising treatment for early stage Alzheimer's disease was approved by the FDA," Brown said. "Therefore, up-to-date information is important for families looking to learn about recent discoveries and available treatments."

The authors of the study write that "the addition of both the source and the date of health-related information and availability in other languages may increase the value of these platforms for both non-medical and medical professionals." It was published in the Journal of Medical Internet Research under the title "ChatGPT vs Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results."

Google and ChatGPT both scored low for readability scores, which makes it difficult for people with lower levels of education and low health literacy skills.

"My prediction is that the readability is the easier thing to improve because there are already some tools, some AI methods, that can read and paraphrase text," Hristidis said. "In terms of improving reliability, accuracy, and so on, that's much harder. Don't forget that it took scientists many decades of AI research to build ChatGPT. It is going to be slow improvements from where we are now."

Hristidis V, Ruggiano N, Brown EL, Ganta SRR, Stewart S.
ChatGPT vs Google for Queries Related to Dementia and Other Cognitive Decline: Comparison of Results.
J Med Internet Res 2023;25:e48966. doi: 10.2196/48966

Most Popular Now

Philips Foundation 2024 Annual Report: E…

Marking its tenth anniversary, Philips Foundation released its 2024 Annual Report, highlighting a year in which the Philips Foundation helped provide access to quality healthcare for 46.5 million people around...

New AI Transforms Radiology with Speed, …

A first-of-its-kind generative AI system, developed in-house at Northwestern Medicine, is revolutionizing radiology - boosting productivity, identifying life-threatening conditions in milliseconds and offering a breakthrough solution to the global radiologist...

Scientists Argue for More FDA Oversight …

An agile, transparent, and ethics-driven oversight system is needed for the U.S. Food and Drug Administration (FDA) to balance innovation with patient safety when it comes to artificial intelligence-driven medical...

Giving Doctors an AI-Powered Head Start …

Detection of melanoma and a range of other skin diseases will be faster and more accurate with a new artificial intelligence (AI) powered tool that analyses multiple imaging types simultaneously...

New Research Finds Specific Learning Str…

If data used to train artificial intelligence models for medical applications, such as hospitals across the Greater Toronto Area, differs from the real-world data, it could lead to patient harm...

AI Agents for Oncology

Clinical decision-making in oncology is challenging and requires the analysis of various data types - from medical imaging and genetic information to patient records and treatment guidelines. To effectively support...

'AI Scientist' Suggests Combin…

An 'AI scientist', working in collaboration with human scientists, has found that combinations of cheap and safe drugs - used to treat conditions such as high cholesterol and alcohol dependence...

Brains vs. Bytes: Study Compares Diagnos…

A University of Maine study compared how well artificial intelligence (AI) models and human clinicians handled complex or sensitive medical cases. The study published in the Journal of Health Organization...

Start-ups in the Spotlight at MEDICA 202…

17 - 20 November 2025, Düsseldorf, Germany. MEDICA, the leading international trade fair and platform for healthcare innovations, will once again confirm its position as the world's number one hotspot for...