The research team developed their foundation model using a comprehensive dataset consisting of 11,467 images of abnormal radiologic scans. Using these images, the model was able to identify patterns that predict anatomical site, malignancy, and prognosis across three different use cases in four cohorts. Compared to existing methods in the field, their approach remained powerful when applied to specialized tasks where only limited data are available. Results are published in Nature Machine Intelligence.
"Given that image biomarker studies are tailored to answer increasingly specific research questions, we believe that our work will enable more accurate and efficient investigations," said first author Suraj Pai from the Artificial Intelligence in Medicine (AIM) Program at Mass General Brigham.
Despite the improved efficacy of AI methods, a key question remains their reliability and explainability (the concept that an AI’s answers can be explained in a way that "makes sense" to humans). The researchers demonstrated that their methods remained stable across inter-reader variations and differences in acquisition. Patterns identified by the foundation model also demonstrated strong associations with underlying biology, mainly correlating with immune-related pathways.
"Our findings demonstrate the efficacy of foundation models in medicine when only limited data might be available for training deep learning networks, especially when applied to identifying reliable imaging biomarkers for cancer-associated use cases," said senior author Hugo Aerts, PhD, director of the AIM Program.
Pai S, Bontempi D, Hadzic I, Prudente V, Sokač M, Chaunzwa TL, Bernatz S, Hosny A, Mak RH, Birkbak NJ, Aerts HJWL.
Foundation model for cancer imaging biomarkers.
Nat Mach Intell. 2024;6(3):354-367. doi: 10.1038/s42256-024-00807-9