New Guidance for Ensuring AI Safety in Clinical Care Published in JAMA

As artificial intelligence (AI) becomes more prevalent in health care, organizations and clinicians must take steps to ensure its safe implementation and use in real-world clinical settings, according to an article co-written by Dean Sittig, PhD, professor with McWilliams School of Biomedical Informatics at UTHealth Houston and Hardeep Singh, MD, MPH, professor at Baylor College of Medicine.

The guidance was published, Nov. 27, 2024, in the Journal of the American Medical Association.

"We often hear about the need for AI to be built safely, but not about how to use it safely in health care settings," Sittig said. "It is a tool that has the potential to revolutionize medical care, but without safeguards in place, AI could generate false or misleading outputs that could potentially harm patients if left unchecked."

Drawing from expert opinion, literature reviews, and experiences with health IT use and safety assessment, Sittig and Singh developed a pragmatic approach for health care organizations and clinicians to monitor and manage AI systems.

"Health care delivery organizations will need to implement robust governance systems and testing processes locally to ensure safe AI and safe use of AI so that ultimately AI can be used to improve the safety of health care and patient outcomes," Singh said. "All health care delivery organizations should check out these recommendations and start proactively preparing for AI now."

Some of the recommended actions for health care organizations are listed below:

  • Review guidance published in high-quality, peer-reviewed journals and conduct rigorous real-world testing to confirm AI’s safety and effectiveness.
  • Establish dedicated committees with multidisciplinary experts to oversee AI system deployment and ensure adherence to safety protocols. Committee members should meet regularly to review requests for new AI applications, consider their safety and effectiveness before implementing them, and develop processes to monitor their performance.
  • Formally train clinicians on AI usage and risk, but also be transparent with patients when AI is part of their care decisions. This transparency is key to building trust and confidence in AI's role in health care.
  • Maintain a detailed inventory of AI systems and regularly evaluate them to identify and mitigate any risks.
  • Develop procedures to turn off AI systems should they malfunction, ensuring smooth transitions back to manual processes.

"Implementing AI into clinical settings should be a shared responsibility among health care providers, AI developers, and electronic health record vendors to protect patients," Sittig said. "By working together, we can build trust and promote the safe adoption of AI in health care."

Sittig DF, Singh H.
Recommendations to Ensure Safety of AI in Real-World Clinical Care.
JAMA. 2024 Nov 27. doi: 10.1001/jama.2024.24598

Most Popular Now

Digital ECGs at Barts Health: A High-Imp…

Opinion Article by Dr Krishnaraj Sinhji Rathod, consultant in interventional cardiology, Barts Health NHS Trust. Picture the moment. A patient in an ambulance, enroute to hospital with new chest pain. Paramedics...

Study Sheds Light on Hurdles Faced in Tr…

Implementing artificial intelligence (AI) into NHS hospitals is far harder than initially anticipated, with complications around governance, contracts, data collection, harmonisation with old IT systems, finding the right AI tools...

Using Deep Learning for Precision Cancer…

Altuna Akalin and his team at the Max Delbrück Center have developed a new tool to more precisely guide cancer treatment. Described in a paper published in Nature Communications, the...

New AI Approach Paves Way for Smarter T-…

Researchers have harnessed the power of artificial intelligence (AI) to tackle one of the most complex challenges in immunology: predicting how T cells recognize and respond to specific peptide antigens...

Study Used AI Models to Improve Predicti…

Chronic kidney disease (CKD) is a complex condition marked by a gradual decline in kidney function, which can ultimately progress to end-stage renal disease (ESRD). Globally, the prevalence of the...

AI-Powered CRISPR could Lead to Faster G…

Stanford Medicine researchers have developed an artificial intelligence (AI) tool to help scientists better plan gene-editing experiments. The technology, CRISPR-GPT, acts as a gene-editing “copilot” supported by AI to help...

Groundbreaking AI Aims to Speed Lifesavi…

To solve a problem, we have to see it clearly. Whether it’s an infection by a novel virus or memory-stealing plaques forming in the brains of Alzheimer’s patients, visualizing disease processes...

AI Spots Hidden Signs of Depression in S…

Depression is one of the most common mental health challenges, but its early signs are often overlooked. It is often linked to reduced facial expressivity. However, whether mild depression or...