AI Model Outperforms Clinicians in Diagnosing Pediatric Ear Infections

An artificial-intelligence (AI) model built at Mass Eye and Ear was shown to be significantly more accurate than doctors at diagnosing pediatric ear infections in the first head-to-head evaluation of its kind, a research team working to develop the model for clinical use reported.

According to a new study published August 16 in Otolaryngology - Head and Neck Surgery, the model, called OtoDX, was more than 95 percent accurate in diagnosing an ear infection in a set of 22 test images compared to 65 percent accuracy among a group of clinicians consisting of ENTs, pediatricians and primary care doctors, who reviewed the same images.

When tested in a dataset of more than 600 inner ear images, the AI model had a diagnostic accuracy of more than 80 percent, representing a significant leap over the average accuracy of clinicians reported in medical literature.

The model utilizes a type of AI called deep learning and was built from hundreds of photographs collected from children prior to undergoing surgery at Mass Eye and Ear for recurrent ear infections or fluid in the ears. The results signify a major step towards the development of a diagnostic tool that can one day be deployed to clinics to assist doctors during patient evaluations, according to the authors. An AI-based diagnostic tool can give providers, like pediatricians and urgent care clinics, an additional test to better inform their clinical decision-making.

"Ear infections are incredibly common in children yet frequently misdiagnosed, leading to delays in care or unnecessary antibiotic prescriptions," said lead study author Matthew Crowson, MD, an otolaryngologist and artificial intelligence researcher at Mass Eye and Ear, and assistant professor of Otolaryngology - Head and Neck Surgery at Harvard Medical School. "This model won’t replace the judgment of clinicians but can serve to supplement their expertise and help them be more confident in their treatment decisions."

Difficult to diagnose common condition

Ear infections occur from a buildup of bacteria inside the middle ear. According to the National Institute on Deafness and Other Communication Disorders, at least five out of six children in the United States have had at least one ear infection before the age of three. When left untreated, ear infections can lead to hearing loss, developmental delays, complications like meningitis, and, in some developing nations, death. Conversely, overtreating children when they don’t have an ear infection can lead to antibiotic resistance and render the medications ineffective against future infections. This latter problem is of significant public health importance.

To ensure the best outcomes for children, clinicians must diagnose ear infections as accurately and early as possible. However, previous studies suggest the conventional diagnostic accuracy of ear infections in children from a physical exam is routinely below 70 percent, even with innovations to technology and clinical practice guidelines. The difficulty of evaluating a child who is struggling or crying during an examination, coupled with the general inexperience many doctors and urgent care providers have in ear evaluations may explain the lower-than-expected diagnostic rate, according to Dr. Crowson.

"Since clinicians would rather stay on the side of caution, it's pretty easy to see why parents typically walk out of urgent care with a prescription for antibiotics," he said.

In 2021, Dr. Crowson collaborated with Mass Eye and Ear colleagues Michael S. Cohen, MD, director of the Multidisciplinary Pediatric Hearing Loss Clinic, and Christopher J. Hartnick, MD, MS, director of the Division of Pediatric Otolaryngology, to develop a more accurate method of diagnosing ear infections using a machine learning algorithm. An artificial neural network was trained with high-resolution, photographs of tympanic membranes collected directly from patients during ear procedures where infection can be seen. These photos represent a gold standard, “ground truth” set of data compared to AI-based tools that rely on images collected from search engines. In a proof-of-concept study published last year, the model was found to be 84 percent accurate in detecting “normal” versus “abnormal” middle ears.

Human versus machine

In the new study, the researchers compared the accuracy of a refined model head-to-head against clinicians. More than 639 images of tympanic membranes from children aged 18 years or younger who were undergoing surgery for tube placement or draining fluid from the ears were used to train the model. The images were tagged as either "normal," "infected," or having "liquid behind the eardrum," as opposed to the "normal" or "abnormal" classification from the team’s earlier model. With the added segment, the model achieved a mean diagnostic accuracy of 80.8 percent.

A survey was then created asking clinicians and trainees of various medical specialties to view 22 new images of tympanic membranes and diagnose the ear as one of the three tagged categories. While the machine-learning model correctly categorized more than 95 percent of the sample images, the average diagnostic score among 39 clinicians who responded to the survey was 65 percent. Moreover, pediatricians and family medicine/general internists correctly categorized 60.1 percent and 59.1 percent of images, respectively.

Bringing artificial intelligence to the clinic

Ongoing studies are underway to validate and refine the AI model. To date, more than 1,000 intraoperative images of tympanic membranes have been amassed at Mass Eye and Ear.

In collaboration with Mass General Brigham Innovation, OtoDx is currently being employed in a prototype device paired with a smartphone app. The device acts as a "mini otoscope" that would fit over the phone's camera and allow clinicians to take photos of the inside of a child's ear, upload them directly to the app and receive a diagnostic reading in seconds. With further validation, OtoDX may provide another tool for clinicians to glean information from in real time during an exam.

As feedback for the pilot is processed, Mass General Brigham Innovation will support the OtoDx team in exploring opportunities to commercialize this impactful tool to assist even more clinicians and their patients.

Crowson MG, Bates DW, Suresh K, Cohen MS, Hartnick CJ.
"Human vs Machine" Validation of a Deep Learning Algorithm for Pediatric Middle Ear Infection Diagnosis.
Otolaryngol Head Neck Surg. 2022 Aug 16:1945998221119156. doi: 10.1177/01945998221119156

Most Popular Now

The Future of Medicine is Data

At the 2023 Annual J.P. Morgan Healthcare Conference, Owkin Co-founder and CEO Thomas Clozel, MD will outline how data is the future of medicine - from the development of new...

Study Surveys Landscape of Apps Built on…

A study led by Regenstrief Institute Research Scientist Titus K. Schleyer, DMD, PhD, is among the first to survey the current landscape of FHIR® apps, providing a snapshot of how...

New Computer Program 'Learns' to Identif…

Genetic mutations cause hundreds of unsolved and untreatable disorders. Among them, DNA mutations in a small percentage of cells, called mosaic mutations, are extremely difficult to detect because they exist...

Applications Open for SpinLab Accelerato…

The start-up accelerator supports entrepreneurial and innovative teams that want to grow sustainably and successfully scale their business model. With a strong hands-on mentality and a lot of passion, the...

Bayer to Accelerate Drug Discovery with …

Bayer AG and Google Cloud today announced a collaboration to drive early drug discovery that will apply Google Cloud's Tensor Processing Units (TPUs), which are custom-developed accelerators designed to run...

Allscripts Announces Corporate Name Chan…

Allscripts Healthcare Solutions, Inc. announced that, effective January 1, 2023, it has changed its name to Veradigm Inc. (NASDAQ: MDRX). Allscripts had been transitioning its solutions to the Veradigm brand...

220M€ Investment in Testing and Experime…

To make the EU the place where AI excellence thrives from the lab to the market, the European Union is setting up world-class Testing and Experimentation Facilities (TEFs) for AI. Together...

AI Tool Developed to Predict Risk of Lun…

Lung cancer is the leading cause of cancer death in the United States and around the world. Low-dose chest computed tomography (LDCT) is recommended to screen people between 50 and...

Artificial Nerve Cells - Almost Like Bio…

Researchers at Linköping University (LiU), Sweden, have created an artificial organic neuron that closely mimics the characteristics of biological nerve cells. This artificial neuron can stimulate natural nerves, making it...

Bayer Acquires Blackford Analysis Ltd. B…

Bayer announced the acquisition of the global strategic imaging AI platform and solutions provider Blackford Analysis Ltd. The acquisition is part of Bayer's strategy to drive innovation in radiology, including...

For Shared Decision-Making, Telemedicine…

Telemedicine may be just as effective as in-person visits when it comes to shared decision-making and communication for patients undergoing a first-time surgery consultation, according to a study published as...

Veradigm Announces Strategic Investment …

Veradigm Inc. (NASDAQ: MDRX), formerly Allscripts Healthcare Solutions, Inc., announced an investment in Holmusk, a global behavioral health real-world evidence and data analytics company, as part of Holmusk's $45 million...