"We are seeing a fundamental shift in how AI tools can be implemented in medicine," says Jakob N. Kather, Professor of Clinical Artificial Intelligence at the EKFZ for Digital Health at TUD and oncologist at the Dresden University Hospital Dresden. "Unlike earlier systems, AI agents are capable of managing complex clinical workflows autonomously. This opens up great opportunities for medicine - but also raises entirely new questions around safety, accountability, and regulation that we need to address," he adds.
Current medical device regulations were designed for static, narrowly focused technologies that maintain human oversight and do not evolve after entering the market. In contrast, new autonomous and broad-scope technologies have fundamentally different characteristics. They demonstrate greater autonomy, adaptability, and scope than previous AI technologies. As they are capable of autonomously executing complex workflows, they present significant challenges for regulators and developers.
"To facilitate the safe and effective implementation of autonomous AI agents in healthcare, regulatory frameworks must evolve beyond static paradigms. We need adaptive regulatory oversight and flexible alternative approval pathways," says Oscar Freyer, lead author of the publication and research associate in the team of Professor Stephen Gilbert, who leads the Medical Device Regulatory Science group at EKFZ for Digital Health, TUD.
As AI agents' capabilities exceed the scope of current regulatory frameworks, the researchers propose several potential solutions to overcome barriers to their implementation.
Immediate adaptations include extending enforcement discretion policies, where regulators acknowledge a product qualifies as a medical device but choose not to enforce certain requirements. Alternatively, a non-medical device classification could be applied to systems that serve a medical purpose but fall outside the traditional medical device regulation.
Medium-term solutions involve developing voluntary alternative pathways (VAPs) and adaptive regulatory frameworks that supplement existing approval processes. These adaptive pathways would shift from static pre-market approval to dynamic oversight using real-world performance data, stakeholder collaboration, and iterative updates. In case of misconduct, devices may be transferred to the established pathways.
The researchers also discuss long-term solutions, such as regulating AI agents in a similar way to the qualification of medical professionals. In this model, regulation would be carried out through structured training processes, in which systems gain autonomy only after demonstrating safe and effective performance.
The researchers note that while tools such as regulatory sandboxes offer some flexibility for testing new technologies, they are not scalable solutions for widespread deployment due to the resources and commitment required by regulatory authorities.
In their article, the researchers argue that meaningful implementation of autonomous AI agents in healthcare will likely remain impossible in the medium term without substantial regulatory reform. VAPs and adaptive pathways are highlighted as the most effective and realistic strategies to achieve this goal. The authors underscore the need for collaborative efforts between regulators, healthcare providers, and technology developers to create frameworks that meet the AI agents' unique characteristics while ensuring patient safety.
"Realizing the full potential of AI agents in healthcare will require bold and forward-thinking reforms," says Stephen Gilbert, Professor of Medical Device Regulatory Science at the EKFZ for Digital Health at TU Dresden and last author of the paper. "Regulators must start preparing now to ensure patient safety and provide clear requirements to enable safe innovation," he adds.
Freyer O, Jayabalan S, Kather JN, Gilbert S.
Overcoming regulatory barriers to the implementation of AI agents in healthcare.
Nat Med. 2025 Jul 18. doi: 10.1038/s41591-025-03841-1