AI for Real-Rime, Patient-Focused Insight

A picture may be worth a thousand words, but still... they both have a lot of work to do to catch up to BiomedGPT.

Covered recently in the prestigious journal Nature Medicine, BiomedGPT is a new a new type of artificial intelligence (AI) designed to support a wide range of medical and scientific tasks. This new study, conducted in collaboration with multiple institutions, is described in the article as "the first open-source and lightweight vision–language foundation model, designed as a generalist capable of performing various biomedical tasks."

"This work combines two types of AI into a decision support tool for medical providers," explains Lichao Sun, an assistant professor of computer science and engineering at Lehigh University and a lead author of the study. "One side of the system is trained to understand biomedical images, and one is trained to understand and assess biomedical text. The combination of these allows the model to tackle a wide range of biomedical challenges, using insight gleaned from databases of biomedical imagery and from the analysis and synthesis of scientific and medical research reports."

The key innovation described in the August 7 Nature Medicine article, “A generalist vision–language foundation model for diverse biomedical tasks,” is that this AI model doesn’t need to be specialized for each task. Typically, AI systems are trained for specific jobs, like recognizing tumors in X-rays or summarizing medical papers. However, this new model can handle many different tasks using the same underlying technology. This versatility makes it a "generalist" model - and a powerful new tool in the hands of medical providers.

"BiomedGPT is based on foundation models, a recent development in AI," says Sun. "Foundation models are large, pre-trained AI systems that can be adapted to various tasks with minimal additional training. The generalist model described in the article has been trained on vast amounts of biomedical data, including images and text, enabling it to perform well across different applications."

"By evaluating 25 datasets across 9 biomedical tasks and different modalities," says Kai Zhang, a Lehigh PhD student advised by Sun who serves as first author of the Nature article, "BiomedGPT achieved 16 state-of-the-art results. A human evaluation of BiomedGPT on three radiology tasks showcased the model’s robust predictive abilities."

Zhang says that he is proud that the open-source codebase is available for other researchers to use as a springboard to drive further development and adoption.

The team reports that the technology behind BiomedGPT may one day help doctors by interpreting complex medical images, assist researchers by analyzing scientific literature, or even aid in drug discovery by predicting how molecules behave.

"The potential impact of such technology is significant," Zhang says, "as it could streamline many aspects of healthcare and research, making them faster and more accurate. Our method demonstrates that effective training with diverse data can lead to more practical biomedical AI for improving diagnosis and workflow efficiency."

A crucial step in the process was validation of the model's effectiveness and applicability in real-world healthcare settings.

"Clinical testing involves applying the AI model to real patient data to assess its accuracy, reliability, and safety," Sun says. "This testing ensures that the model performs well across different scenarios. The outcomes of these tests helped refine the model, demonstrating its potential to improve clinical decision-making and patient care."

Massachusetts General Hospital (MGH), a founding member of the Mass General Brigham healthcare system and teaching affiliate of Harvard Medical School, played a crucial role in the development and validation of the BiomedGPT model. The institution's involvement primarily focused on providing clinical expertise and facilitating the evaluation of the model's effectiveness in real-world healthcare settings. For instance, the model was tested with radiologists at MGH, where it demonstrated superior performance in tasks like visual question answering and radiology report generation. This collaboration helped ensure that the model was both accurate and practical for clinical use.

Other contributors to BiomedGPT include researchers from University of Georgia, Samsung Research America, University of Pennsylvania, Stanford University, University of Central Florida, UC-Santa Cruz, University of Texas-Health, Children’s Hospital of Philadelphia, and the Mayo Clinic.

"This research is highly interdisciplinary and collaborative," says Sun. "The research involves expertise from multiple fields, including computer science, medicine, radiology, and biomedical engineering. Each author contributes specialized knowledge necessary to develop, test, and validate the model across various biomedical tasks. Large-scale projects like this often require access to diverse datasets and computational resources, along with access to skills in algorithm development, model training, evaluation, and application to real-world scenarios, as well as clinical testing and validation.

"This was a true team effort," he says. "Creating something that can truly help the medical community improve patient outcomes across a wide range of issues is a very complex challenge. With such complexity, collaboration is key to creating impact through the application of science and engineering."

Zhang K, Zhou R, Adhikarla E, Yan Z, Liu Y, Yu J, Liu Z, Chen X, Davison BD, Ren H, Huang J, Chen C, Zhou Y, Fu S, Liu W, Liu T, Li X, Chen Y, He L, Zou J, Li Q, Liu H, Sun L.
A generalist vision-language foundation model for diverse biomedical tasks.
Nat Med. 2024 Aug 7. doi: 10.1038/s41591-024-03185-2

Most Popular Now

AI-Powered CRISPR could Lead to Faster G…

Stanford Medicine researchers have developed an artificial intelligence (AI) tool to help scientists better plan gene-editing experiments. The technology, CRISPR-GPT, acts as a gene-editing “copilot” supported by AI to help...

Groundbreaking AI Aims to Speed Lifesavi…

To solve a problem, we have to see it clearly. Whether it’s an infection by a novel virus or memory-stealing plaques forming in the brains of Alzheimer’s patients, visualizing disease processes...

AI Spots Hidden Signs of Depression in S…

Depression is one of the most common mental health challenges, but its early signs are often overlooked. It is often linked to reduced facial expressivity. However, whether mild depression or...

AI Model Forecasts Disease Risk Decades …

Imagine a future where your medical history could help predict what health conditions you might face in the next two decades. Researchers have developed a generative AI model that uses...

AI Tools Help Predict Severe Asthma Risk…

Mayo Clinic researchers have developed artificial intelligence (AI) tools that help identify which children with asthma face the highest risk of serious asthma exacerbation and acute respiratory infections. The study...

AI Model Indicates Four out of Ten Breas…

A project at Lund University in Sweden has trained an AI model to identify breast cancer patients who could be spared from axillary surgery. The model analyses previously unutilised information...

Smart Device Uses AI and Bioelectronics …

As a wound heals, it goes through several stages: clotting to stop bleeding, immune system response, scabbing, and scarring. A wearable device called "a-Heal," designed by engineers at the University...

AI Distinguishes Glioblastoma from Look-…

A Harvard Medical School–led research team has developed an AI tool that can reliably tell apart two look-alike cancers found in the brain but with different origins, behaviors, and treatments. The...

ChatGPT 4o Therapeutic Chatbot 'Ama…

One of the first randomized controlled trials assessing the effectiveness of a large language model (LLM) chatbot 'Amanda' for relationship support shows that a single session of chatbot therapy...

Overcoming the AI Applicability Crisis a…

Opinion Article by Harry Lykostratis, Chief Executive, Open Medical. The government’s 10 Year Health Plan makes a lot of the potential of AI-software to support clinical decision making, improve productivity, and...

Dartford and Gravesham Implements Clinis…

Dartford and Gravesham NHS Trust has taken a significant step towards a more digital future by rolling out electronic test ordering using Clinisys ICE. The trust deployed the order communications...