Paper Calls for Patient-First Regulation of AI in Healthcare

Ever wonder if the latest and greatest artificial intelligence (AI) tool you read about in the morning paper is going to save your life? A new study published in JAMA led by John W. Ayers, Ph.D., of the Qualcomm Institute within the University of California San Diego, finds that question can be difficult to answer since AI products in healthcare do not universally undergo any externally evaluated approval process assessing how it might benefit patient outcomes before coming to market.

The research team evaluated the recent White House Executive Order that instructed the Department of Health and Human Services to develop new AI-specific regulatory strategies addressing equity, safety, privacy, and quality for AI in healthcare before April 27, 2024. However, team members were surprised to find the order did not once mention patient outcomes, the standard metric by which healthcare products are judged before being allowed to access the healthcare marketplace.

"The goal of medicine is to save lives," said Davey Smith, M.D., head of the Division of Infectious Disease and Global Public Health at UC San Diego School of Medicine, co-director of the university’s Altman Clinical and Translational Research Institute, and study senior author. "AI tools should prove clinically significant improvements in patient outcomes before they are widely adopted."

According to the team, AI-powered early warning systems for sepsis, a fatal acute illness among hospitalized patients that affects 1.7 million Americans each year, demonstrates the consequences of inadequate prioritization of patient outcomes in regulations. A third-party evaluation of the most widely adopted AI sepsis prediction model revealed 67% of patients who developed sepsis were not identified by the system. Would hospital administrators have chosen this sepsis prediction system if trials assessing patient outcomes data were mandated, the team wondered, considering the array of available early warning systems for sepsis?

"We are calling for a revision to the White House Executive Order that prioritizes patient outcomes when regulating AI products," added John W. Ayers, Ph.D., who is deputy director of informatics in Altman Clinical and Translational Research Institute in addition to his Qualcomm Institute affiliation. "Similar to pharmaceutical products, AI tools that impact patient care should be evaluated by federal agencies for how they improve patients' feeling, function, and survival."

The team points to its 2023 study in JAMA Internal Medicine on using AI-powered chatbots to respond to patient messages as an example of what patient outcome-centric regulations can achieve. "Theoretically, a study comparing standard care versus standard care enhanced by AI conversational agents might find differences in downstream care utilization in some patient populations, such as heart failure patients," said Nimit Desai, B.S., who is a research affiliate at the Qualcomm Institute, UC San Diego School of Medicine student, and study coauthor. "But studies like this don’t just happen unless regulators appropriately incentivize them. With a patient outcomes-centric approach, AI for patient messaging and all other clinical applications can truly enhance people’s lives."

The team recognizes that its proposed regulatory strategy can be a significant lift for AI and healthcare industry partners and may not be necessary for every flavor of AI use case in healthcare. However, the researchers say, excluding patient outcomes-centric rules in the White House Executive Order is a serious omission.

Ayers JW, Desai N, Smith DM.
Regulate Artificial Intelligence in Health Care by Prioritizing Patient Outcomes.
JAMA. 2024 Jan 29. doi: 10.1001/jama.2024.0549

Most Popular Now

Transforming Drug Discovery with AI

A new AI-powered program will allow researchers to level up their drug discovery efforts. The program, called TopoFormer, was developed by an interdisciplinary team led by Guowei Wei, a Michigan...

We may Soon be Able to Detect Cancer wit…

A new paper in Biology Methods & Protocols, published by Oxford University Press, indicates that it may soon be possible for doctors to use artificial intelligence (AI) to detect and...

Maternity Tech Launched to Help NHS Meas…

Health tech provider C2-Ai has formally launched a new 'observatory' system to help hospitals gain a better understanding of risks, outcomes and safety within maternity and neonatal services. Announced at the...

Large Language Models Illuminate a Progr…

This study is led by Prof. Bin Dong (Beijing International Center for Mathematical Research, Peking University) and Prof. Lin Shen (Department of Gastrointestinal Oncology, Key Laboratory of Carcinogenesis and Translational...

Health Innovation East Partners with Cog…

Health Innovation East, the innovation arm of the NHS in the East of England and Cogniss, a no-code ecosystem for digital health solutions, have announced a strategic partnership to launch...

An AI-Powered Wearable System Tracks the…

Scientists at the University of Southern California have developed an artificial intelligence (AI)-powered system to track tiny devices that monitor markers of disease in the gut. Devices using the novel...

"Self-Taught" AI Tool Helps to…

A computer program based on data from nearly a half-million tissue images and powered by artificial intelligence (AI) can accurately diagnose cases of adenocarcinoma, the most common form of lung...

New Computational Model of Real Neurons …

Nearly all the neural networks that power modern artificial intelligence (AI) tools such as ChatGPT are based on a 1960s-era computational model of a living neuron. A new model developed...

Meet CARMEN, a Robot that Helps People w…

Meet CARMEN, short for Cognitively Assistive Robot for Motivation and Neurorehabilitation - a small, tabletop robot designed to help people with mild cognitive impairment (MCI) learn skills to improve memory...

AI Matches Protein Interaction Partners

Proteins are the building blocks of life, involved in virtually every biological process. Understanding how proteins interact with each other is crucial for deciphering the complexities of cellular functions, and...

AI Model to Improve Patient Response to …

A new artificial intelligence (AI) tool that can help to select the most suitable treatment for cancer patients has been developed by researchers at The Australian National University (ANU). DeepPT, developed...

Mobile Phone Data Helps Track Pathogen S…

A new way to map the spread and evolution of pathogens, and their responses to vaccines and antibiotics, will provide key insights to help predict and prevent future outbreaks. The...