Google Research Shows How AI can Make Ophthalmologists More Effective

As artificial intelligence continues to evolve, diagnosing disease faster and potentially with greater accuracy than physicians, some have suggested that technology may soon replace tasks that physicians currently perform. But a new study from the Google AI research group shows that physicians and algorithms working together are more effective than either alone. It's one of the first studies to examine how AI can improve physicians' diagnostic accuracy. The new research will be published in the April edition of Ophthalmology, the ournal of the American Academy of Ophthalmology.

This study expands on previous work from Google AI showing that its algorithm works roughly as well as human experts in screening patients for a common diabetic eye disease called diabetic retinopathy. For their latest study, the researchers wanted to see if their algorithm could do more than simply diagnose disease. They wanted to create a new computer-assisted system that could "explain" the algorithm's diagnosis. They found that this system not only improved the ophthalmologists' diagnostic accuracy, but it also improved algorithm's accuracy.

More than 29 million Americans have diabetes, and are at risk for diabetic retinopathy, a potentially blinding eye disease. People typically don't notice changes in their vision in the disease's early stages. But as it progresses, diabetic retinopathy usually causes vision loss that in many cases cannot be reversed. That's why it's so important that people with diabetes have yearly screenings.

Unfortunately, the accuracy of screenings can vary significantly. One study found a 49 percent error rate among internists, diabetologists, and medical residents.

Recent advances in AI promise to improve access to diabetic retinopathy screening and to improve its accuracy. But it's less clear how AI will work in the physician's office or other clinical settings. Previous attempts to use computer-assisted diagnosis shows that some screeners rely on the machine too much, which leads to repeating the machine's errors, or under-rely on it and ignore accurate predictions. Researchers at Google AI believe some of these pitfalls may be avoided if the computer can "explain" its predictions.

To test this theory, the researchers developed two types of assistance to help physicians read the algorithm's predictions.

  • Grades: A set of five scores that represent the strength of evidence for the algorithm's prediction.
  • Grades + heatmap: Enhance the grading system with a heatmap that measures the contribution of each pixel in the image to the algorithm's prediction.

Ten ophthalmologists (four general ophthalmologists, one trained outside the US, four retina specialists, and one retina specialist in training) were asked to read each image once under one of three conditions: unassisted, grades only, and grades + heatmap.

Both types of assistance improved physicians' diagnostic accuracy. It also improved their confidence in the diagnosis. But the degree of improvement depended on the physician's level of expertise.

Without assistance, general ophthalmologists are significantly less accurate than the algorithm, while retina specialists are not significantly more accurate than the algorithm. With assistance, general ophthalmologists match but do not exceed the model's accuracy, while retina specialists start to exceed the model's performance.

"What we found is that AI can do more than simply automate eye screening, it can assist physicians in more accurately diagnosing diabetic retinopathy," said lead researcher, Rory Sayres, PhD.. "AI and physicians working together can be more accurate than either alone."

Like medical technologies that preceded it, Sayres said that AI is another tool that will make the knowledge, skill, and judgment of physicians even more central to quality care.

"There's an analogy in driving," Sayres explained. "There are self-driving vehicles, and there are tools to help drivers, like Android Auto. The first is automation, the second is augmentation. The findings of our study indicate that there may be space for augmentation in classifying medical images like retinal fundus images. When the combination of clinician and assistant outperforms either alone, this provides an argument for up-leveling clinicians with intelligent tools."

Rory Sayres, Ankur Taly, Ehsan Rahimy, Katy Blumer, David Coz, Naama Hammel, Jonathan Krause, Arunachalam Narayanaswamy, Zahra Rastegar, Derek Wu, Shawn Xu, Scott Barb, Anthony Joseph, Michael Shumski, Jesse Smith, Arjun B Sood, Greg S Corrado, Lily Peng, Dale R Webster.
Using a Deep Learning Algorithm and Integrated Gradients Explanationto Assist Grading for Diabetic Retinopathy.
Ophthalmology, Volume 126, Issue 4, 552 - 564. doi: 10.1016/j.ophtha.2018.11.016.

Most Popular Now

People may Trust Computers more than Hum…

Despite increasing concern over the intrusion of algorithms in daily life, people may be more willing to trust a computer program than their fellow humans, especially if a task becomes...

Alcidion Acquires UK Patient Flow Softwa…

Smart health technology provider Alcidion Group has today announced the strategic acquisition of ExtraMed, a UK company which specialises in providing NHS trusts with software that helps frontline staff make...

Microsoft Accelerates Industry Cloud Str…

Microsoft Corp (Nasdaq: MSFT) and Nuance Communications, Inc. (Nasdaq: NUAN) have entered into a definitive agreement under which Microsoft will acquire Nuance for $56.00 per share, implying a 23% premium...

Expediting the Diagnosis of Heart Attack…

Siemens Healthineers announced today the company's Atellica VTLi Patient-Side Immunoassay Analyzer(1) has obtained CE mark and is expected to be available later this spring. The Atellica VTLi analyzer provides lab...

New AI Tool Tracks Evolution of COVID-19…

A new machine-learning program accurately identifies COVID-19-related conspiracy theories on social media and models how they evolved over time - a tool that could someday help public health officials combat...

Siemens Healthineers Completes Acquisiti…

Siemens Healthineers AG (Frankfurt: SHL) announced that it has successfully completed the acquisition of Varian Medical Systems, Inc. ("Varian"). The acquisition was previously announced on August 2, 2020. "With Varian, Siemens...

Talking Health Tech

Opinion Article by Andy Kinnear, Partnerships Director, Ethical Healthcare Consulting. Ethical is different. The guy who founded it, [chief executive] Thomas Webb, comes from an NHS background and he is very...

Commission Opens First European Innovati…

Following the launch of the European Innovation Council (EIC) and the announcement of the first funding opportunities, today the Commission opened the first EIC Accelerator calls. The funding worth over...

Artificial Intelligence Model Predicts w…

The human immune defense is based on the ability of white blood cells to accurately identify disease-causing pathogens and to initiate a defense reaction against them. The immune defense is...

Call for Papers: DigiHealthDay-2021

12 November 2021 The DigiHealthDay 2021 is an International Symposium with the theme "Global Digital Health - Today, Tomorrow, and Beyond". The abstracts pertaining to the field of digital health and...

International Master in Digital Health i…

The Master of Digital Health, MDH (formerly - Master of Medical Informatics, MMI) at European Campus Rottal-Inn (ECRI) - a branch of the Deggendorf University of Applied Sciences (DIT) in Bavaria...

Proposal for a Regulation Laying Down Ha…

Artificial Intelligence (AI) is a fast evolving family of technologies that can bring a wide array of economic and societal benefits across the entire spectrum of industries and social activities...