AI Matches or Outperforms Human Specialists in Retina and Glaucoma Management

A large language model (LLM) artificial intelligence (AI) system can match, or in some cases outperform, human ophthalmologists in the diagnosis and treatment of patients with glaucoma and retina disease, according to research from New York Eye and Ear Infirmary of Mount Sinai (NYEE).

The provocative study, published February 22, in JAMA Ophthalmology, suggests that advanced AI tools, which are trained on vast amounts of data, text, and images, could play an important role in providing decision-making support to ophthalmologists in the diagnosis and management of cases involving glaucoma and retina disorders, which afflict millions of patients.

The study matched the knowledge of ophthalmic specialists against the capabilities of the latest generation AI system, GPT-4 (Generative Pre-Training–Model 4) from OpenAI, designed to replicate human-level performance. Within medicine, sophisticated AI tools are seen as potentially revolutionizing diagnosis and treatment tools through the accuracy and comprehensiveness of their LLM-generated responses. Ophthalmology, with its high volume of often complex patients, could be a particularly fertile field for AI, giving specialists more time to practice evidence-based medicine.

"The performance of GPT-4 in our study was quite eye-opening," says Andy Huang, MD, an ophthalmology resident at NYEE, and lead author of the study. "We recognized the enormous potential of this AI system from the moment we started testing it and were fascinated to observe that GPT-4 could not only assist but in some cases match or exceed, the expertise of seasoned ophthalmic specialists."

For the human side of its study, the Mount Sinai team recruited 12 attending specialists and three senior trainees from the Department of Ophthalmology at the Icahn School of Medicine at Mount Sinai. A basic set of 20 questions (10 each for glaucoma and retina) from the American Academy of Ophthalmology’s list of commonly asked questions by patients was randomly selected, along with 20 deidentified patient cases culled from Mount Sinai-affiliated eye clinics. Responses from both the GPT-4/AI system and human specialists were then statistically analyzed and rated for accuracy and thoroughness using a Likert scale, which is commonly used in clinical research to score responses.

The results showed that AI matched or outperformed human specialists in both accuracy and completeness of its medical advice and assessments. More specifically, AI demonstrated superior performance in response to glaucoma questions and case-management advice, while reflecting a more balanced outcome in retina questions, where AI matched humans in accuracy but exceeded them in completeness.

"AI was particularly surprising in its proficiency in handling both glaucoma and retina patient cases, matching the accuracy and completeness of diagnoses and treatment suggestions made by human doctors in a clinical note format," says Louis R. Pasquale, MD, FARVO, Deputy Chair for Ophthalmology Research for the Department of Ophthalmology, and senior author of the study. "Just as the AI application Grammarly can teach us how to be better writers, GPT-4 can give us valuable guidance on how to be better clinicians, especially in terms of how we document findings of patient exams."

While emphasizing that additional testing is needed, Dr. Huang believes this work points to a promising future for AI in ophthalmology. "It could serve as a reliable assistant to eye specialists by providing diagnostic support and potentially easing their workload, especially in complex cases or areas of high patient volume," he explains. "For patients, the integration of AI into mainstream ophthalmic practice could result in quicker access to expert advice, coupled with more informed decision-making to guide their treatment."

Huang AS, Hirabayashi K, Barna L, Parikh D, Pasquale LR.
Assessment of a Large Language Model's Responses to Questions and Cases About Glaucoma and Retina Management.
JAMA Ophthalmol. 2024 Feb 22:e236917. doi: 10.1001/jamaophthalmol.2023.6917

Most Popular Now

Unlocking the 10 Year Health Plan

The government's plan for the NHS is a huge document. Jane Stephenson, chief executive of SPARK TSL, argues the key to unlocking its digital ambitions is to consider what it...

Alcidion Grows Top Talent in the UK, wit…

Alcidion has today announced the addition of three new appointments to their UK-based team, with one internal promotion and two external recruits. Dr Paul Deffley has been announced as the...

AI can Find Cancer Pathologists Miss

Men assessed as healthy after a pathologist analyses their tissue sample may still have an early form of prostate cancer. Using AI, researchers at Uppsala University have been able to...

AI, Full Automation could Expand Artific…

Automated insulin delivery (AID) systems such as the UVA Health-developed artificial pancreas could help more type 1 diabetes patients if the devices become fully automated, according to a new review...

How AI could Speed the Development of RN…

Using artificial intelligence (AI), MIT researchers have come up with a new way to design nanoparticles that can more efficiently deliver RNA vaccines and other types of RNA therapies. After training...

MIT Researchers Use Generative AI to Des…

With help from artificial intelligence, MIT researchers have designed novel antibiotics that can combat two hard-to-treat infections: drug-resistant Neisseria gonorrhoeae and multi-drug-resistant Staphylococcus aureus (MRSA). Using generative AI algorithms, the research...

AI Hybrid Strategy Improves Mammogram In…

A hybrid reading strategy for screening mammography, developed by Dutch researchers and deployed retrospectively to more than 40,000 exams, reduced radiologist workload by 38% without changing recall or cancer detection...

Penn Developed AI Tools and Datasets Hel…

Doctors treating kidney disease have long depended on trial-and-error to find the best therapies for individual patients. Now, new artificial intelligence (AI) tools developed by researchers in the Perelman School...

New Training Year Starts at Siemens Heal…

In September, 197 school graduates will start their vocational training or dual studies in Germany at Siemens Healthineers. 117 apprentices and 80 dual students will begin their careers at Siemens...

Are You Eligible for a Clinical Trial? C…

A new study in the academic journal Machine Learning: Health discovers that ChatGPT can accelerate patient screening for clinical trials, showing promise in reducing delays and improving trial success rates. Researchers...

New AI Tool Addresses Accuracy and Fairn…

A team of researchers at the Icahn School of Medicine at Mount Sinai has developed a new method to identify and reduce biases in datasets used to train machine-learning algorithms...

Global Study Reveals How Patients View M…

How physicians feel about artificial intelligence (AI) in medicine has been studied many times. But what do patients think? A team led by researchers at the Technical University of Munich...