In the days before artificial intelligence (AI) and machine learning (ML), clinicians performed this crucial yet painstaking and time-consuming task by hand, but over the past decade, U-nets - a type of AI architecture specifically designed for medical image segmentation - have been the go-to instead. However, U-nets require large amounts of data and resources to be trained.
"For large and/or 3D images, these demands are costly," said Kushal Vyas, a Rice electrical and computer engineering doctoral student and first author on a paper presented at the Medical Image Computing and Computer Assisted Intervention Society, or MICCAI, the leading conference in the field. "In this study, we proposed MetaSeg, a completely new way of performing image segmentation."
In experiments using 2D and 3D brain magnetic resonance imaging (MRI) data, MetaSeg was shown to achieve the same segmentation performance as U-Nets while needing 90% fewer parameters - the key variables AI/ML models derive from training data and use to identify patterns and make predictions.
The study, titled "Fit Pixels, Get Labels: Meta-learned Implicit Networks for Image Segmentation," won the best paper award at MICCAI, getting recognized from a pool of over 1,000 accepted submissions.
"Instead of U-Nets, MetaSeg leverages implicit neural representations - a neural network framework that has hitherto not been thought useful or explored for image segmentation," Vyas said.
An implicit neural representation (INR) is an AI network that interprets a medical image as a mathematical formula that accounts for the signal value (color, brightness, etc.) of each and every pixel in a 2D image or every voxel in a 3D one.
While INRs offer a very detailed yet compact way to represent information, they are also highly specific, meaning they typically only work well for the single signal/image they trained on: An INR trained on a brain MRI cannot typically generalize rules about what different parts of the brain look like, so if provided with an image of a different brain, the INR would typically falter.
"INRs have been used in the computer vision and medical imaging communities for tasks such as 3D scene reconstruction and signal compression, which only require modeling one signal at a time," Vyas said. "However, it was not obvious before MetaSeg how to use them for tasks such as segmentation, which require learning patterns over many signals."
To make it useful for medical image segmentation, the researchers taught INRs to predict both the signal values and the specific segmentation labels for a given image. To do so, they used meta-learning, an AI training strategy whose literal translation is "learning to learn" that helps models rapidly adapt to new information.
"We prime the INR model parameters in such a way so that they are further optimized on an unseen image at test time, which enables the model to decode the image features into accurate labels," Vyas said.
This special training allows the INRs to not only quickly adjust themselves to match the pixels or voxels of a previously unseen medical image but to then also decode its labels, instantly predicting where the outlines for different anatomical regions should go.
"MetaSeg offers a fresh, scalable perspective to the field of medical image segmentation that has been dominated for a decade by U-Nets," said Guha Balakrishnan, assistant professor of electrical and computer engineering at Rice and a member of the university’s Ken Kennedy Institute. "Our research results promise to make medical image segmentation far more cost-effective while delivering top performance."
Vyas K, Veeraraghavan A, Balakrishnan G.
Fit Pixels, Get Labels: Meta-learned Implicit Networks for Image Segmentation. In: Gee, J.C., et al. Medical Image Computing and Computer Assisted Intervention - MICCAI 2025.
MICCAI 2025. Lecture Notes in Computer Science, vol 15962. Springer, Cham. 2025. doi: 10.1007/978-3-032-04947-6_19