Proposed Framework for Integrating Chatbots into Health Care

While the technology for developing artificial intelligence-powered chatbots has existed for some time, a new viewpoint piece in JAMA lays out the clinical, ethical, and legal aspects that must be considered before applying them in healthcare. And while the emergence of COVID-19 and the social distancing that accompanies it has prompted more health systems to explore and apply automated chatbots, the authors still urge caution and thoughtfulness before proceeding.

"We need to recognize that this is relatively new technology and even for the older systems that were in place, the data are limited," said the viewpoint's lead author, John D. McGreevey III, MD, an associate professor of Medicine in the Perelman School of Medicine at the University of Pennsylvania. "Any efforts also need to realize that much of the data we have comes from research, not widespread clinical implementation. Knowing that, evaluation of these systems must be robust when they enter the clinical space, and those operating them should be nimble enough to adapt quickly to feedback."

McGreevey, joined by C. William Hanson III, MD, chief medical information officer at Penn Medicine, and Ross Koppel, PhD, FACMI, a senior fellow at the Leonard Davis Institute of Healthcare Economics at Penn and professor of Medical Informatics, wrote "Clinical, Legal, and Ethical Aspects of AI-Assisted Conversational Agents." In it, the authors lay out 12 different focus areas that should be considered when planning to implement a chatbot, or, more formally, "conversational agent," in clinical care.

Chatbots are a tool used to communicate with patients via text message or voice. Many chatbots are powered by artificial intelligence (AI). This paper specifically discusses chatbots that use natural language processing, an AI process that seeks to "understand" language used in conversations and draws threads and connections from them to provide meaningful and useful answers.

With health care, those messages, and people's reactions to them, are extremely important and carry tangible consequences.

"We are increasingly in direct communication with our patients through electronic medical records, giving them direct access to their test results, diagnoses and doctors' notes," Hanson said. "Chatbots have the ability to enhance the value of those communications on the one hand, or cause confusion or even harm, on the other."

For instance, how a chatbot handles someone telling it something as serious as "I want to hurt myself" has many different implications.

In the self-harm example, there are several areas of focus laid out by the authors that apply. This touches first and foremost on the "Patient Safety" category: Who monitors the chatbot and how often do they do it? It also touches on "Trust and Transparency": Would this patient actually take a response from a known chatbot seriously? It also, unfortunately, raises questions in the paper's "Legal & Licensing" category: Who is accountable if the chatbot fails in its task. Moreover, a question under the "Scope" category may apply here, too: Is this a task best suited for a chatbot, or is it something that should still be totally human-operated?

Within their viewpoint, the team believes they have laid out key considerations that can inform a framework for decision-making when it comes to implementing chatbots in health care. Their considerations should apply even when rapid implementation is required to respond to events like the spread of COVID-19.

"To what extent should chatbots be extending the capabilities of clinicians, which we'd call augmented intelligence, or replacing them through totally artificial intelligence?" Koppel said. "Likewise, we need to determine the limits of chatbot authority to perform in different clinical scenarios, such as when a patient indicates that they have a cough, should the chatbot only respond by letting a nurse know or digging in further: 'Can you tell me more about your cough?'"

Chatbots have the opportunity to significantly improve health outcomes and lower health systems' operating costs, but evaluation and research will be key to that: both to ensure smooth operation and to keep the trust of both patients and health care workers.

"It's our belief that the work is not done when the conversational agent is deployed," McGreevey said. "These are going to be increasingly impactful technologies that deserve to be monitored not just before they are launched, but continuously throughout the life cycle of their work with patients."

Na-Na Zhang, Xiao-Feng Li, Yong-Qiang Deng, Hui Zhao, Yi-Jiao Huang, GuanYang, Wei-Jin Huang, Peng Gao, Chao Zhou, Rong-Rong Zhang, Yan Guo, Shi-Hui Sun, Hang Fan, Shu-Long Zu, Qi Chen, Qi He, Tian-Shu Cao, Xing-Yao Huang,Hong-Ying Qiu, Jian-Hui Nie, Yuhang Jiang, Hua-Yuan Yan, Qing Ye, Xia Zhong,Xia-Lin Xue, Zhen-Yu Zha, Dongsheng Zhou, Xiao Yang, You-Chun Wang, Bo Ying,Cheng-Feng Qin.
A thermostable mRNA vaccine against COVID-19.
Cell, 2020. doi: 10.1016/j.cell.2020.07.024

Most Popular Now

Contact Tracing Apps Unlikely to Contain…

Contract tracing apps used to reduce the spread of COVID-19 are unlikely to be effective without proper uptake and support from concurrent control measures, finds a new study by UCL...

Philips Launches New Innovations to Help…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, today presented new approaches to health and healthcare, and the shift to health-at-home, at its virtual consumer health...

Internet Searches for Anxiety Attacks Ta…

Many health experts are concerned that the COVID-19 pandemic could be having widespread effects on people's mental health, but assessing these concerns is difficult without data. "Traditional public health surveillance lacks...

Orion Health Secures Place on Emerging T…

Remote Patient Monitoring solution included on Crown Commercial Services Spark: Technology Innovation Marketplace. Healthcare organisations now have a simple route to procure personalised monitoring for people living with chronic conditions to...

Scientists Develop Low-Cost Chip to Dete…

Robust and widespread antibody testing has emerged as a key strategy in the fight against SARS-CoV-2, the virus responsible for the COVID-19 pandemic. However current testing methods are too inaccurate...

Newly Merged CCG Expands Tech Support to…

Norfolk and Waveney Clinical Commissioning Group to support prescribers to improve the quality and safety of prescribing through deploying prescribing decision technology FDB OptimiseRx. A newly merged clinical commissioning group...

Anonymized Cell Phone Location Data can …

In March 2020, federal officials declared the COVID-19 outbreak a national emergency. Around the same time, most states implemented stay-at-home advisories - to different degrees and at different times. Publicly...

Designed Antiviral Proteins Inhibit SARS…

Computer-designed small proteins have now been shown to protect lab-grown human cells from SARS-CoV-2, the coronavirus that causes COVID-19. The findings are reported today in Science. In the experiments, the lead...

Telehealth Scales Up During the Pandemic…

Hailed for its ability to erase distance between health care providers in cities and patients in rural areas, telehealth has ironically enabled medical care to continue in a time when...

Fitbit Receives Regulatory Clearance in …

Fitbit (NYSE:FIT) has received 510(k) clearance from the U.S. Food and Drug Administration (FDA), as well as Conformité Européenne (CE) marking in the European Union, for its electrocardiogram (ECG) app...

Don't Fall for the 'Next Best Action' Tr…

15 September, 2020. London, UK. OKRA has announced a new, free webinar called 'Why Next Best Action is not enough', a panel will discuss the urgent need to move beyond the...

Cerner Chosen to Support Population Heal…

Cerner Corporation is pleased to announce a new relationship with Hampshire and Isle of Wight (HIOW) health and care system to enable their population health management journey to transform the...