AI Hybrid Strategy Improves Mammogram Interpretation

A hybrid reading strategy for screening mammography, developed by Dutch researchers and deployed retrospectively to more than 40,000 exams, reduced radiologist workload by 38% without changing recall or cancer detection rates. The study, which emphasizes AI confidence, was published in Radiology, a journal of the Radiological Society of North America (RSNA).

"Although the overall performance of state-of-the-art AI models is very high, AI sometimes makes mistakes," said Sarah D. Verboom, M.Sc., a doctoral candidate in the Department of Medical Imaging at Radboud University Medical Center in the Netherlands. "Identifying exams in which AI interpretation is unreliable is crucial to allow for and optimize use of AI models in breast cancer screening programs."

The hybrid reading strategy involves using a combination of radiologist readers and a stand-alone AI interpretation of cases in which the AI model performs as well as, or better than, the radiologist.

"We can achieve this performance level if the AI model provides not only an assessment of the probability of malignancy (PoM) for a case but also a rating of its certainty of that assessment," Verboom said. "Unfortunately, the PoM itself is not always a good predictor of certainty because deep neural networks tend to be overconfident in their predictions."

To develop and evaluate a hybrid reading strategy, the researchers used a dataset of 41,469 screening mammography exams from 15,522 women (median age 59 years) with 332 screen-detected cancers and 34 interval cancers. The exams were performed between 2003 and 2018 in Utrecht, Netherlands, as part of the Dutch National Breast Cancer Screening Program.

The dataset was divided at the patient level into two equal groups with identical cancer detection, recall and interval cancer rates. The first group was used to determine the optimal thresholds for the hybrid reading strategy, while the second group was used to evaluate the reading strategies.

Of the uncertainty metrics evaluated by the researchers, the entropy of the mean PoM score of the most suspicious region produced a cancer detection rate of 6.6 per 1,000 cases and a recall rate of 23.7 per 1,000 cases, similar to rates of standard double-reading by radiologists.

The final hybrid reading strategy involved AI evaluating every screening mammogram to produce two outputs: the PoM and an uncertainty estimate of that prediction. When AI determined the PoM was below the established threshold with certainty, the case was considered normal. When AI detected a PoM above the established threshold, women were recalled for further testing, but only when that prediction was deemed confident. Otherwise, the exam was double-read by radiologists.

Although the majority of AI decisions were uncertain and deferred to a human reader, 38% were classified as certain and could be read solely by AI. Using the researchers’ strategy reduced radiologist reading workload to 61.9% without changing recall (23.6‰ vs 23.9‰) or cancer detection (6.6‰ vs 6.7‰) rates, both of which are comparable to those of standard double-reading.

When the AI model was certain, the area under the curve (AUC) was higher (0.96 vs 0.87). Its sensitivity nearly matched that of double radiologist reading (85.4% vs 88.9%). Younger women with dense breasts were more likely to have an uncertain AI score.

"The key component of our study isn’t necessarily that this is the best way to split the workload, but that it’s helpful to have uncertainty quantification built into AI models," Verboom said. "I hope commercial products integrate this into their models, because I think it’s a very useful metric."

Verboom noted that if the study results occurred in clinical practice, the decision to recall 19% of women would be made by AI without the intervention of a radiologist.

"Several studies have shown that women participating in breast cancer screening programs have positive attitudes about the use of AI," she said. "However, most women prefer their mammogram to be read by at least one radiologist."

She said it may be more acceptable for radiologists to review exams deemed uncertain by AI, as well as AI recall cases.

"The use of AI with uncertainty quantification can be a possible solution for workforce shortages and could help build trust in the implementation of AI," Verboom said.

Verboom said further research, ideally a prospective trial, is needed to determine how the workload reduction achieved by the hybrid reading strategy could decrease radiologist reading time.

"I think in the future, we could get to a point where a portion of women are sent home without ever having a radiologist look at their mammogram because AI will determine that their exam is normal," she said. "We’re not there yet, but I think we could get there with this uncertainty metric and quality control."

This study is part of the aiREAD project, which is financed by the Dutch Research Council, Dutch Cancer Society and Health Holland.

Verboom SD, Kroes J, Pires S, Broeders MJM, Sechopoulos I.
AI Should Read Mammograms Only When Confident: A Hybrid Breast Cancer Screening Reading Strategy.
Radiology. 2025 Aug;316(2):e242594. doi: 10.1148/radiol.242594

Most Popular Now

Unlocking the 10 Year Health Plan

The government's plan for the NHS is a huge document. Jane Stephenson, chief executive of SPARK TSL, argues the key to unlocking its digital ambitions is to consider what it...

Alcidion Grows Top Talent in the UK, wit…

Alcidion has today announced the addition of three new appointments to their UK-based team, with one internal promotion and two external recruits. Dr Paul Deffley has been announced as the...

New Training Year Starts at Siemens Heal…

In September, 197 school graduates will start their vocational training or dual studies in Germany at Siemens Healthineers. 117 apprentices and 80 dual students will begin their careers at Siemens...

AI can Find Cancer Pathologists Miss

Men assessed as healthy after a pathologist analyses their tissue sample may still have an early form of prostate cancer. Using AI, researchers at Uppsala University have been able to...

AI, Full Automation could Expand Artific…

Automated insulin delivery (AID) systems such as the UVA Health-developed artificial pancreas could help more type 1 diabetes patients if the devices become fully automated, according to a new review...

New AI Tool Addresses Accuracy and Fairn…

A team of researchers at the Icahn School of Medicine at Mount Sinai has developed a new method to identify and reduce biases in datasets used to train machine-learning algorithms...

Are You Eligible for a Clinical Trial? C…

A new study in the academic journal Machine Learning: Health discovers that ChatGPT can accelerate patient screening for clinical trials, showing promise in reducing delays and improving trial success rates. Researchers...

Global Study Reveals How Patients View M…

How physicians feel about artificial intelligence (AI) in medicine has been studied many times. But what do patients think? A team led by researchers at the Technical University of Munich...

AI Analysis of Colonoscopy Improves Asse…

In a new study, artificial intelligence (AI) matched and potentially exceeded the performance of gastroenterologists and conventional scoring in evaluating endoscopies of Crohn’s disease patients. The results, published in Clinical Gastroenterology...

International Study Reveals Sex and Age …

An international research team led by Assistant Professor Zhiyu Wan from ShanghaiTech University has recently published groundbreaking findings in the journal Health Data Science, highlighting biases in multimodal large language...

Digital ECGs at Barts Health: A High-Imp…

Opinion Article by Dr Krishnaraj Sinhji Rathod, consultant in interventional cardiology, Barts Health NHS Trust. Picture the moment. A patient in an ambulance, enroute to hospital with new chest pain. Paramedics...

Study Sheds Light on Hurdles Faced in Tr…

Implementing artificial intelligence (AI) into NHS hospitals is far harder than initially anticipated, with complications around governance, contracts, data collection, harmonisation with old IT systems, finding the right AI tools...