AI Model Outperforms Clinicians in Diagnosing Pediatric Ear Infections

An artificial-intelligence (AI) model built at Mass Eye and Ear was shown to be significantly more accurate than doctors at diagnosing pediatric ear infections in the first head-to-head evaluation of its kind, a research team working to develop the model for clinical use reported.

According to a new study published August 16 in Otolaryngology - Head and Neck Surgery, the model, called OtoDX, was more than 95 percent accurate in diagnosing an ear infection in a set of 22 test images compared to 65 percent accuracy among a group of clinicians consisting of ENTs, pediatricians and primary care doctors, who reviewed the same images.

When tested in a dataset of more than 600 inner ear images, the AI model had a diagnostic accuracy of more than 80 percent, representing a significant leap over the average accuracy of clinicians reported in medical literature.

The model utilizes a type of AI called deep learning and was built from hundreds of photographs collected from children prior to undergoing surgery at Mass Eye and Ear for recurrent ear infections or fluid in the ears. The results signify a major step towards the development of a diagnostic tool that can one day be deployed to clinics to assist doctors during patient evaluations, according to the authors. An AI-based diagnostic tool can give providers, like pediatricians and urgent care clinics, an additional test to better inform their clinical decision-making.

"Ear infections are incredibly common in children yet frequently misdiagnosed, leading to delays in care or unnecessary antibiotic prescriptions," said lead study author Matthew Crowson, MD, an otolaryngologist and artificial intelligence researcher at Mass Eye and Ear, and assistant professor of Otolaryngology - Head and Neck Surgery at Harvard Medical School. "This model won’t replace the judgment of clinicians but can serve to supplement their expertise and help them be more confident in their treatment decisions."

Difficult to diagnose common condition

Ear infections occur from a buildup of bacteria inside the middle ear. According to the National Institute on Deafness and Other Communication Disorders, at least five out of six children in the United States have had at least one ear infection before the age of three. When left untreated, ear infections can lead to hearing loss, developmental delays, complications like meningitis, and, in some developing nations, death. Conversely, overtreating children when they don’t have an ear infection can lead to antibiotic resistance and render the medications ineffective against future infections. This latter problem is of significant public health importance.

To ensure the best outcomes for children, clinicians must diagnose ear infections as accurately and early as possible. However, previous studies suggest the conventional diagnostic accuracy of ear infections in children from a physical exam is routinely below 70 percent, even with innovations to technology and clinical practice guidelines. The difficulty of evaluating a child who is struggling or crying during an examination, coupled with the general inexperience many doctors and urgent care providers have in ear evaluations may explain the lower-than-expected diagnostic rate, according to Dr. Crowson.

"Since clinicians would rather stay on the side of caution, it's pretty easy to see why parents typically walk out of urgent care with a prescription for antibiotics," he said.

In 2021, Dr. Crowson collaborated with Mass Eye and Ear colleagues Michael S. Cohen, MD, director of the Multidisciplinary Pediatric Hearing Loss Clinic, and Christopher J. Hartnick, MD, MS, director of the Division of Pediatric Otolaryngology, to develop a more accurate method of diagnosing ear infections using a machine learning algorithm. An artificial neural network was trained with high-resolution, photographs of tympanic membranes collected directly from patients during ear procedures where infection can be seen. These photos represent a gold standard, “ground truth” set of data compared to AI-based tools that rely on images collected from search engines. In a proof-of-concept study published last year, the model was found to be 84 percent accurate in detecting “normal” versus “abnormal” middle ears.

Human versus machine

In the new study, the researchers compared the accuracy of a refined model head-to-head against clinicians. More than 639 images of tympanic membranes from children aged 18 years or younger who were undergoing surgery for tube placement or draining fluid from the ears were used to train the model. The images were tagged as either "normal," "infected," or having "liquid behind the eardrum," as opposed to the "normal" or "abnormal" classification from the team’s earlier model. With the added segment, the model achieved a mean diagnostic accuracy of 80.8 percent.

A survey was then created asking clinicians and trainees of various medical specialties to view 22 new images of tympanic membranes and diagnose the ear as one of the three tagged categories. While the machine-learning model correctly categorized more than 95 percent of the sample images, the average diagnostic score among 39 clinicians who responded to the survey was 65 percent. Moreover, pediatricians and family medicine/general internists correctly categorized 60.1 percent and 59.1 percent of images, respectively.

Bringing artificial intelligence to the clinic

Ongoing studies are underway to validate and refine the AI model. To date, more than 1,000 intraoperative images of tympanic membranes have been amassed at Mass Eye and Ear.

In collaboration with Mass General Brigham Innovation, OtoDx is currently being employed in a prototype device paired with a smartphone app. The device acts as a "mini otoscope" that would fit over the phone's camera and allow clinicians to take photos of the inside of a child's ear, upload them directly to the app and receive a diagnostic reading in seconds. With further validation, OtoDX may provide another tool for clinicians to glean information from in real time during an exam.

As feedback for the pilot is processed, Mass General Brigham Innovation will support the OtoDx team in exploring opportunities to commercialize this impactful tool to assist even more clinicians and their patients.

Crowson MG, Bates DW, Suresh K, Cohen MS, Hartnick CJ.
"Human vs Machine" Validation of a Deep Learning Algorithm for Pediatric Middle Ear Infection Diagnosis.
Otolaryngol Head Neck Surg. 2022 Aug 16:1945998221119156. doi: 10.1177/01945998221119156

Most Popular Now

AI-Powered CRISPR could Lead to Faster G…

Stanford Medicine researchers have developed an artificial intelligence (AI) tool to help scientists better plan gene-editing experiments. The technology, CRISPR-GPT, acts as a gene-editing “copilot” supported by AI to help...

Groundbreaking AI Aims to Speed Lifesavi…

To solve a problem, we have to see it clearly. Whether it’s an infection by a novel virus or memory-stealing plaques forming in the brains of Alzheimer’s patients, visualizing disease processes...

AI Spots Hidden Signs of Depression in S…

Depression is one of the most common mental health challenges, but its early signs are often overlooked. It is often linked to reduced facial expressivity. However, whether mild depression or...

ChatGPT 4o Therapeutic Chatbot 'Ama…

One of the first randomized controlled trials assessing the effectiveness of a large language model (LLM) chatbot 'Amanda' for relationship support shows that a single session of chatbot therapy...

AI Tools Help Predict Severe Asthma Risk…

Mayo Clinic researchers have developed artificial intelligence (AI) tools that help identify which children with asthma face the highest risk of serious asthma exacerbation and acute respiratory infections. The study...

AI Model Forecasts Disease Risk Decades …

Imagine a future where your medical history could help predict what health conditions you might face in the next two decades. Researchers have developed a generative AI model that uses...

AI Distinguishes Glioblastoma from Look-…

A Harvard Medical School–led research team has developed an AI tool that can reliably tell apart two look-alike cancers found in the brain but with different origins, behaviors, and treatments. The...

AI Model Indicates Four out of Ten Breas…

A project at Lund University in Sweden has trained an AI model to identify breast cancer patients who could be spared from axillary surgery. The model analyses previously unutilised information...

Smart Device Uses AI and Bioelectronics …

As a wound heals, it goes through several stages: clotting to stop bleeding, immune system response, scabbing, and scarring. A wearable device called "a-Heal," designed by engineers at the University...

Overcoming the AI Applicability Crisis a…

Opinion Article by Harry Lykostratis, Chief Executive, Open Medical. The government’s 10 Year Health Plan makes a lot of the potential of AI-software to support clinical decision making, improve productivity, and...

Dartford and Gravesham Implements Clinis…

Dartford and Gravesham NHS Trust has taken a significant step towards a more digital future by rolling out electronic test ordering using Clinisys ICE. The trust deployed the order communications...