Proposed Framework for Integrating Chatbots into Health Care

While the technology for developing artificial intelligence-powered chatbots has existed for some time, a new viewpoint piece in JAMA lays out the clinical, ethical, and legal aspects that must be considered before applying them in healthcare. And while the emergence of COVID-19 and the social distancing that accompanies it has prompted more health systems to explore and apply automated chatbots, the authors still urge caution and thoughtfulness before proceeding.

"We need to recognize that this is relatively new technology and even for the older systems that were in place, the data are limited," said the viewpoint's lead author, John D. McGreevey III, MD, an associate professor of Medicine in the Perelman School of Medicine at the University of Pennsylvania. "Any efforts also need to realize that much of the data we have comes from research, not widespread clinical implementation. Knowing that, evaluation of these systems must be robust when they enter the clinical space, and those operating them should be nimble enough to adapt quickly to feedback."

McGreevey, joined by C. William Hanson III, MD, chief medical information officer at Penn Medicine, and Ross Koppel, PhD, FACMI, a senior fellow at the Leonard Davis Institute of Healthcare Economics at Penn and professor of Medical Informatics, wrote "Clinical, Legal, and Ethical Aspects of AI-Assisted Conversational Agents." In it, the authors lay out 12 different focus areas that should be considered when planning to implement a chatbot, or, more formally, "conversational agent," in clinical care.

Chatbots are a tool used to communicate with patients via text message or voice. Many chatbots are powered by artificial intelligence (AI). This paper specifically discusses chatbots that use natural language processing, an AI process that seeks to "understand" language used in conversations and draws threads and connections from them to provide meaningful and useful answers.

With health care, those messages, and people's reactions to them, are extremely important and carry tangible consequences.

"We are increasingly in direct communication with our patients through electronic medical records, giving them direct access to their test results, diagnoses and doctors' notes," Hanson said. "Chatbots have the ability to enhance the value of those communications on the one hand, or cause confusion or even harm, on the other."

For instance, how a chatbot handles someone telling it something as serious as "I want to hurt myself" has many different implications.

In the self-harm example, there are several areas of focus laid out by the authors that apply. This touches first and foremost on the "Patient Safety" category: Who monitors the chatbot and how often do they do it? It also touches on "Trust and Transparency": Would this patient actually take a response from a known chatbot seriously? It also, unfortunately, raises questions in the paper's "Legal & Licensing" category: Who is accountable if the chatbot fails in its task. Moreover, a question under the "Scope" category may apply here, too: Is this a task best suited for a chatbot, or is it something that should still be totally human-operated?

Within their viewpoint, the team believes they have laid out key considerations that can inform a framework for decision-making when it comes to implementing chatbots in health care. Their considerations should apply even when rapid implementation is required to respond to events like the spread of COVID-19.

"To what extent should chatbots be extending the capabilities of clinicians, which we'd call augmented intelligence, or replacing them through totally artificial intelligence?" Koppel said. "Likewise, we need to determine the limits of chatbot authority to perform in different clinical scenarios, such as when a patient indicates that they have a cough, should the chatbot only respond by letting a nurse know or digging in further: 'Can you tell me more about your cough?'"

Chatbots have the opportunity to significantly improve health outcomes and lower health systems' operating costs, but evaluation and research will be key to that: both to ensure smooth operation and to keep the trust of both patients and health care workers.

"It's our belief that the work is not done when the conversational agent is deployed," McGreevey said. "These are going to be increasingly impactful technologies that deserve to be monitored not just before they are launched, but continuously throughout the life cycle of their work with patients."

Na-Na Zhang, Xiao-Feng Li, Yong-Qiang Deng, Hui Zhao, Yi-Jiao Huang, GuanYang, Wei-Jin Huang, Peng Gao, Chao Zhou, Rong-Rong Zhang, Yan Guo, Shi-Hui Sun, Hang Fan, Shu-Long Zu, Qi Chen, Qi He, Tian-Shu Cao, Xing-Yao Huang,Hong-Ying Qiu, Jian-Hui Nie, Yuhang Jiang, Hua-Yuan Yan, Qing Ye, Xia Zhong,Xia-Lin Xue, Zhen-Yu Zha, Dongsheng Zhou, Xiao Yang, You-Chun Wang, Bo Ying,Cheng-Feng Qin.
A thermostable mRNA vaccine against COVID-19.
Cell, 2020. doi: 10.1016/j.cell.2020.07.024

Most Popular Now

AI can Detect COVID-19 in the Lungs Like…

A University of Central Florida researcher is part of a new study showing that artificial intelligence can be nearly as accurate as a physician in diagnosing COVID-19 in the lungs. The...

New Research Reveals Impact of COVID-19 …

Hospitals face new demand for virtual visiting, and more patients are confident they don't always need to see a doctor in person, as the use of remote technologies rise. But...

Telemedicine Saves Chronic Pain Patients…

Patients who saw a pain medicine specialist via telemedicine saved time and money and were highly satisfied with their experience, even before the COVID-19 pandemic, according to a study being...

Black Country Pathology Services Deliver…

Black Country Pathology Services has pushed ahead with a major initiative to create a laboratory network to serve four trusts and delivered a successful go-live for its CliniSys laboratory information...

Greater Manchester and Sectra Sign Deal …

Eight NHS hospital trusts in Greater Manchester are to benefit from medical imaging technology that will transform how doctors and other NHS healthcare professionals access and review images from patient...

North Bristol NHS Clinicians Go Live in …

North Bristol NHS Trust has successfully and safely fast-tracked the roll out of a series of digital tools to clinicians to help in their response to COVID-19. Over 3,500 clinicians went...

The Walton Centre Extends 17-Year-Long R…

The Walton Centre NHS Foundation Trust has renewed its contract with Silverlink Software, to retain the use of one of the country's leading patient administration systems (PAS), supporting the trust’s...

Allscripts Continues to Lead in Interope…

Allscripts (NASDAQ: MDRX) remains committed to effective interoperability in healthcare delivery. A leader in interoperability and open platforms, the company has made it a priority to deliver clinicians accessible, clinically...

Philips Partners with LeQuest to Provide…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, and LeQuest, a leading provider of online interactive simulation-based training for the use of medical devices [1], today...

New Fluoroscopy System from Siemens Heal…

Siemens Healthineers is debuting its new fluoroscopy system, the Luminos Lotus Max, at the annual French JFR 2020. The system offers industry-leading technology in radiography and fluoroscopy. "Combining these state-of-the-art...

Depression in Teenagers Prompts Pioneeri…

Teenagers starting their GCSE and A level studies are being treated for depression through a new digital mental health therapy programme for young people that reflects the increasing education pressures...