Scientists Argue for More FDA Oversight of Healthcare AI Tools

An agile, transparent, and ethics-driven oversight system is needed for the U.S. Food and Drug Administration (FDA) to balance innovation with patient safety when it comes to artificial intelligence-driven medical technologies. That is the takeaway from a new report issued to the FDA, published in the open-access journal PLOS Medicine by Leo Celi of the Massachusetts Institute of Technology, and colleagues.

Artificial intelligence (AI) is becoming a powerful force in healthcare, helping doctors diagnose diseases, monitor patients, and even recommend treatments. Unlike traditional medical devices, many AI tools continue to learn and change after they’ve been approved, meaning their behavior can shift in unpredictable ways once they’re in use.

In the new paper, Celi and his colleagues argue that the FDA's current system is not set up to keep tabs on these post-approval changes. Their analysis calls for stronger rules around transparency and bias, especially to protect vulnerable populations. If an algorithm is trained mostly on data from one group of people, it may make mistakes when used with others. The authors recommend that developers be required to share information about how their AI models were trained and tested, and that the FDA involve patients and community advocates more directly in decision-making. They also suggest practical fixes, including creating public data repositories to track how AI performs in the real world, offering tax incentives for companies that follow ethical practices, and training medical students to critically evaluate AI tools.

"This work has the potential to drive real-world impact by prompting the FDA to rethink existing oversight mechanisms for AI-enabled medical technologies. We advocate for a patient-centered, risk-aware, and continuously adaptive regulatory approach - one that ensures AI remains an asset to clinical practice without compromising safety or exacerbating healthcare disparities," the authors say.

Abulibdeh R, Celi LA, Sejdić E.
The illusion of safety: A report to the FDA on AI healthcare product approvals.
PLOS Digit Health. 2025 Jun 5;4(6):e0000866. doi: 10.1371/journal.pdig.0000866

Most Popular Now

Giving Doctors an AI-Powered Head Start …

Detection of melanoma and a range of other skin diseases will be faster and more accurate with a new artificial intelligence (AI) powered tool that analyses multiple imaging types simultaneously...

AI Agents for Oncology

Clinical decision-making in oncology is challenging and requires the analysis of various data types - from medical imaging and genetic information to patient records and treatment guidelines. To effectively support...

AI Medical Receptionist Modernizing Doct…

A virtual medical receptionist named "Cassie," developed through research at Texas A&M University, is transforming the way patients interact with health care providers. Cassie is a digital-human assistant created by Humanate...

Using Data and AI to Create Better Healt…

Academic medical centers could transform patient care by adopting principles from learning health systems principles, according to researchers from Weill Cornell Medicine and the University of California, San Diego. In...

AI Tool Set to Transform Characterisatio…

A multinational team of researchers, co-led by the Garvan Institute of Medical Research, has developed and tested a new AI tool to better characterise the diversity of individual cells within...

AI Detects Hidden Heart Disease Using Ex…

Mass General Brigham researchers have developed a new AI tool in collaboration with the United States Department of Veterans Affairs (VA) to probe through previously collected CT scans and identify...

Human-AI Collectives Make the Most Accur…

Diagnostic errors are among the most serious problems in everyday medical practice. AI systems - especially large language models (LLMs) like ChatGPT-4, Gemini, or Claude 3 - offer new ways...

Northern Ireland Completes Nationwide Ro…

Go-lives at Western and Southern health and social care trusts mean every pathology service is using the same laboratory information management system; improving efficiency and quality. An ambitious technology project to...

Highland Marketing Announced as Official…

Highland Marketing has been named, for the second year running, the official communications partner for HETT Show 2025, the UK's leading digital health conference and exhibition. Taking place 7-8 October...

MHP-Net: A Revolutionary AI Model for Ac…

Liver cancer is the sixth most common cancer globally and a leading cause of cancer-related deaths. Accurate segmentation of liver tumors is a crucial step for the management of the...

Groundbreaking TACIT Algorithm Offers Ne…

Researchers at VCU Massey Comprehensive Cancer Center have developed a novel algorithm that could provide a revolutionary tool for determining the best options for patients - both in the treatment...

The Many Ways that AI Enters Rheumatolog…

High-resolution computed tomography (HRCT) is the standard to diagnose and assess progression in interstitial lung disease (ILD), a key feature in systemic sclerosis (SSc). But AI-assisted interpretation has the potential...