ChatGPT is Debunking Myths on Social Media Around Vaccine Safety

ChatGPT could help to increase vaccine uptake by debunking myths around jab safety, say the authors of a study published in the peer-reviewed journal Human Vaccines and Immunotherapeutics.

The researchers asked the artificial intelligence (AI) chatbot the top 50 most frequently-asked COVID-19 vaccine questions. They included queries based on myths and fake stories such as the vaccine causing Long COVID.

Results show that ChatGPT scored nine out of 10 on average for accuracy. The rest of the time it was correct but left some gaps in the information provided, according to the study.

Based on these findings, experts who led the study from the GenPoB research group based at the Instituto de Investigación Sanitaria (IDIS) - Hospital Clinico Universitario of Santiago de Compostela, say the AI tool is a "reliable source of non-technical information to the public", especially for people without specialist scientific knowledge.

However, the findings do highlight some concerns about the technology such as ChatGPT changing its answers in certain situations.

"Overall, ChatGPT constructs a narrative in line with the available scientific evidence, debunking myths circulating on social media," says lead author Antonio Salas, who as well as leading the GenPoB research group, is also a Professor at the Faculty of Medicine at the University of Santiago de Compostela, in Spain.

"Thereby it potentially facilitates an increase in vaccine uptake. ChatGPT can detect counterfeit questions related to vaccines and vaccination. The language this AI uses is not too technical and therefore easily understandable to the public but without losing scientific rigor.

"We acknowledge that the present-day version of ChatGPT cannot substitute an expert or scientific evidence. But the results suggest it could be a reliable source of information to the public."

In 2019, the World Health Organisation (WHO) listed vaccine hesitancy among the top 10 threats to global health.

During the pandemic, misinformation spread via social media contributed to public mistrust of COVID-19 vaccination.

The authors of this study include those from the Hospital Clinico Universitario de Santiago which the WHO designated as a vaccine safety collaborating center in 2018.

Researchers at the center have been exploring myths around vaccine safety and medical situations that are falsely believed to be a reason not to get vaccinated. These misplaced concerns contribute to vaccine hesitancy.

The study authors set out to test ChatGPT's ability to get the facts right and share accurate information around Covid vaccine safety in line with current scientific evidence.

ChatGPT enables people to have human-like conversations and interactions with a virtual assistant. The technology is very user-friendly which makes it accessible to a wide population.

However, many governments are concerned about the potential for ChatGPT to be used fraudulently in educational settings such as universities.

The study was designed to challenge the chatbot by asking it the questions most frequently received by the WHO collaborating center in Santiago.

The queries covered three themes. The first was misconceptions around safety such as the vaccine causing Long COVID. Next was false contraindications - medical situations where the jab is safe to use such as in breastfeeding women.

The questions also related to true contraindications - a health condition where the vaccine should not be used - and cases where doctors must take precautions e.g. a patient with heart muscle inflammation.

Next, experts analyzed the responses then rated them for veracity and precision against current scientific evidence, and recommendations from WHO and other international agencies.

The authors say this was important because algorithms created by social media and internet search engines are often based on an individual's usual preferences. This may lead to 'biased or wrong answers', they add.

Results showed that most of the questions were answered correctly with an average score of nine out of 10 which is defined as 'excellent' or 'good'. The responses to the three question themes were on average 85.5% accurate or 14.5% accurate but with gaps in the information provided by ChatGPT.

ChatGPT provided correct answers to queries that arose from genuine vaccine myths, and to those considered in clinical recommendation guidelines to be false or true contraindications.

However, the research team does highlight ChatGPT's downsides in providing vaccine information.

Professor Salas, who specializes in human genetics, concludes: "ChatGPT provides different answers if the question is repeated 'with a few seconds of delay'.

"Another concern we have seen is that this AI tool, in its present version, could also be trained to provide answers not in line with scientific evidence.

"One can 'torture' the system in such a way that it will provide the desired answer. This is also true for other contexts different to vaccines. For instance, it might be possible to make the chatbot align with absurd narratives like the flat-earth theory, deny climate change, or object to the theory of evolution, just to give a few examples.

"However, it's important to note that these responses are not the default behavior of ChatGPT. Thus, the results we have obtained regarding vaccine safety can be probably extrapolated to many other myths and pseudoscience."

Salas A, Rivero-Calle I, Martinón-Torres F.
Chatting with ChatGPT to learn about safety of COVID-19 vaccines - A perspective.
Hum Vaccin Immunother. 2023 Aug 1;19(2):2235200. doi: 10.1080/21645515.2023.2235200

Most Popular Now

AI Predictions for Colorectal Cancer: On…

Colorectal cancer (CRC) ranks second in leading causes of cancer-related deaths globally, according to the WHO. For the first time, researchers from Helmholtz Munich and the University of Technology Dresden...

Combining AI Models Improves Breast Canc…

Combining artificial intelligence (AI) systems for short- and long-term breast cancer risk results in an improved cancer risk assessment, according to a study published in Radiology, a journal of the...

ChatGPT Shows 'Impressive' Acc…

A new study led by investigators from Mass General Brigham has found that ChatGPT was about 72 percent accurate in overall clinical decision making, from coming up with possible diagnoses...

Healthcare Chatbot: Expand Support with …

The Danish eHealth platform, sundhed.dk, has faced a substantial surge in requests from Danish citizens and has swiftly expanded its support and effectively adapt to the ongoing changes in queries due...

WiFi SPARK's Healthcare Business Re…

Leading WiFi provider WiFi SPARK is rebranding its healthcare arm as SPARK Technology Services Limited. The new identity marks the completion of the integration of the former Hospedia bedside unit...

ChatGPT is Debunking Myths on Social Med…

ChatGPT could help to increase vaccine uptake by debunking myths around jab safety, say the authors of a study published in the peer-reviewed journal Human Vaccines and Immunotherapeutics. The researchers asked...

Online AI-Based Test for Parkinson'…

An artificial intelligence (AI) tool developed by researchers at the University of Rochester can help people with Parkinson's disease remotely assess the severity of their symptoms within minutes. A study...

AI Performs Comparably to Human Readers …

Using a standardized assessment, researchers in the UK compared the performance of a commercially available artificial intelligence (AI) algorithm with human readers of screening mammograms. Results of their findings were...

Siemens Healthineers Expands Production …

Siemens Healthineers is expanding its site in Rudolstadt, Germany. By mid 2024, a new manufacturing building will be built on the site. The new manufacturing plant will produce electron accelerators...

More Cases of Breast Cancer Detected wit…

One radiologist supported by AI detected more cases of breast cancer in screening mammography than two radiologists working together, reports the ScreenTrustCAD study from Karolinska Institutet in The Lancet Digital...

MEDICA 2023 + COMPAMED 2023: "Where…

13 - 16 November 2023, Düsseldorf, Germany. The medical technology market is in worldwide motion and the signs ahead of MEDICA 2023 and COMPAMED 2023 in Düsseldorf as the internationally leading...

Smartphone Technology Expected to Advanc…

Since the 1980s, we have known that neurological soft signs (NSS) can distinguish people with schizophrenia from psychiatrically healthy individuals. NSS are subtle neurological impairments that principally manifest as decreased...