AI Hybrid Strategy Improves Mammogram Interpretation

A hybrid reading strategy for screening mammography, developed by Dutch researchers and deployed retrospectively to more than 40,000 exams, reduced radiologist workload by 38% without changing recall or cancer detection rates. The study, which emphasizes AI confidence, was published in Radiology, a journal of the Radiological Society of North America (RSNA).

"Although the overall performance of state-of-the-art AI models is very high, AI sometimes makes mistakes," said Sarah D. Verboom, M.Sc., a doctoral candidate in the Department of Medical Imaging at Radboud University Medical Center in the Netherlands. "Identifying exams in which AI interpretation is unreliable is crucial to allow for and optimize use of AI models in breast cancer screening programs."

The hybrid reading strategy involves using a combination of radiologist readers and a stand-alone AI interpretation of cases in which the AI model performs as well as, or better than, the radiologist.

"We can achieve this performance level if the AI model provides not only an assessment of the probability of malignancy (PoM) for a case but also a rating of its certainty of that assessment," Verboom said. "Unfortunately, the PoM itself is not always a good predictor of certainty because deep neural networks tend to be overconfident in their predictions."

To develop and evaluate a hybrid reading strategy, the researchers used a dataset of 41,469 screening mammography exams from 15,522 women (median age 59 years) with 332 screen-detected cancers and 34 interval cancers. The exams were performed between 2003 and 2018 in Utrecht, Netherlands, as part of the Dutch National Breast Cancer Screening Program.

The dataset was divided at the patient level into two equal groups with identical cancer detection, recall and interval cancer rates. The first group was used to determine the optimal thresholds for the hybrid reading strategy, while the second group was used to evaluate the reading strategies.

Of the uncertainty metrics evaluated by the researchers, the entropy of the mean PoM score of the most suspicious region produced a cancer detection rate of 6.6 per 1,000 cases and a recall rate of 23.7 per 1,000 cases, similar to rates of standard double-reading by radiologists.

The final hybrid reading strategy involved AI evaluating every screening mammogram to produce two outputs: the PoM and an uncertainty estimate of that prediction. When AI determined the PoM was below the established threshold with certainty, the case was considered normal. When AI detected a PoM above the established threshold, women were recalled for further testing, but only when that prediction was deemed confident. Otherwise, the exam was double-read by radiologists.

Although the majority of AI decisions were uncertain and deferred to a human reader, 38% were classified as certain and could be read solely by AI. Using the researchers’ strategy reduced radiologist reading workload to 61.9% without changing recall (23.6‰ vs 23.9‰) or cancer detection (6.6‰ vs 6.7‰) rates, both of which are comparable to those of standard double-reading.

When the AI model was certain, the area under the curve (AUC) was higher (0.96 vs 0.87). Its sensitivity nearly matched that of double radiologist reading (85.4% vs 88.9%). Younger women with dense breasts were more likely to have an uncertain AI score.

"The key component of our study isn’t necessarily that this is the best way to split the workload, but that it’s helpful to have uncertainty quantification built into AI models," Verboom said. "I hope commercial products integrate this into their models, because I think it’s a very useful metric."

Verboom noted that if the study results occurred in clinical practice, the decision to recall 19% of women would be made by AI without the intervention of a radiologist.

"Several studies have shown that women participating in breast cancer screening programs have positive attitudes about the use of AI," she said. "However, most women prefer their mammogram to be read by at least one radiologist."

She said it may be more acceptable for radiologists to review exams deemed uncertain by AI, as well as AI recall cases.

"The use of AI with uncertainty quantification can be a possible solution for workforce shortages and could help build trust in the implementation of AI," Verboom said.

Verboom said further research, ideally a prospective trial, is needed to determine how the workload reduction achieved by the hybrid reading strategy could decrease radiologist reading time.

"I think in the future, we could get to a point where a portion of women are sent home without ever having a radiologist look at their mammogram because AI will determine that their exam is normal," she said. "We’re not there yet, but I think we could get there with this uncertainty metric and quality control."

This study is part of the aiREAD project, which is financed by the Dutch Research Council, Dutch Cancer Society and Health Holland.

Verboom SD, Kroes J, Pires S, Broeders MJM, Sechopoulos I.
AI Should Read Mammograms Only When Confident: A Hybrid Breast Cancer Screening Reading Strategy.
Radiology. 2025 Aug;316(2):e242594. doi: 10.1148/radiol.242594

Most Popular Now

AI also Assesses Dutch Mammograms Better…

AI is detecting tumors more often and earlier in the Dutch breast cancer screening program. Those tumors can then be treated at an earlier stage. This has been demonstrated by...

RSNA AI Challenge Models can Independent…

Algorithms submitted for an AI Challenge hosted by the Radiological Society of North America (RSNA) have shown excellent performance for detecting breast cancers on mammography images, increasing screening sensitivity while...

AI could Help Emergency Rooms Predict Ad…

Artificial intelligence (AI) can help emergency department (ED) teams better anticipate which patients will need hospital admission, hours earlier than is currently possible, according to a multi-hospital study by the...

Head-to-Head Against AI, Pharmacy Studen…

Students pursuing a Doctor of Pharmacy degree routinely take - and pass - rigorous exams to prove competency in several areas. Can ChatGPT accurately answer the same questions? A new...

NHS Active 10 Walking Tracker Users are …

Users of the NHS Active 10 app, designed to encourage people to become more active, immediately increased their amount of brisk and non-brisk walking upon using the app, according to...

New AI Tool Illuminates "Dark Side…

Proteins sustain life as we know it, serving many important structural and functional roles throughout the body. But these large molecules have cast a long shadow over a smaller subclass...

Brain Imaging may Identify Patients Like…

By understanding differences in how people’s brains are wired, clinicians may be able to predict who would benefit from a self-guided anxiety care app, according to a new analysis from...

The Human Touch of Doctors will Still be…

AI-based medicine will revolutionise care including for Alzheimer’s and diabetes, predicts a technology expert, but it must be accessible to all patients. Healing with Artificial Intelligence, written by technology expert Daniele...

Deep Learning-Based Model Enables Fast a…

Stroke is the second leading cause of death globally. Ischemic stroke, strongly linked to atherosclerotic plaques, requires accurate plaque and vessel wall segmentation and quantification for definitive diagnosis. However, conventional...

Stepping for Digital Rewards

Walking is well known to have significant health benefits, but few people achieve the daily recommended steps. Fortunately, mobile health (mHealth) applications have emerged as promising tools to promote physical...

AI can Find Cancer Pathologists Miss

Men assessed as healthy after a pathologist analyses their tissue sample may still have an early form of prostate cancer. Using AI, researchers at Uppsala University have been able to...

Unlocking the 10 Year Health Plan

The government's plan for the NHS is a huge document. Jane Stephenson, chief executive of SPARK TSL, argues the key to unlocking its digital ambitions is to consider what it...