Augmented Reality System Lets Doctors See under Patients' Skin without the Scalpel

New technology is bringing the power of augmented reality into clinical practice. The system, called ProjectDR, allows medical images such as CT scans and MRI data to be displayed directly on a patient's body in a way that moves as the patient does.

"We wanted to create a system that would show clinicians a patient's internal anatomy within the context of the body," explained Ian Watts, a computing science graduate student and the developer of ProjectDR.

The technology includes a motion-tracking system using infrared cameras and markers on the patient's body, as well as a projector to display the images. But the really difficult part, Watts explained, is having the image track properly on the patient's body even as they shift and move. The solution: custom software written by Watts that gets all of the components working together.

Vast applications

"There are lots of applications for this technology, including in teaching, physiotherapy, laparoscopic surgery and even surgical planning," said Watts, who developed the technology with fellow graduate student Michael Fiest.

ProjectDR also has the capacity to present segmented images - for example, only the lungs or only the blood vessels - depending on what a clinician is interested in seeing.

For now, Watts is working on refining ProjectDR to improve the system's automatic calibration and to add components such as depth sensors. The next steps are testing the program's viability in a clinical setting, explained Pierre Boulanger, professor in the Department of Computing Science.

Next steps

"Soon, we'll deploy ProjectDR in an operating room in a surgical simulation laboratory to test the pros and cons in real-life surgical applications," said Boulanger. "We are also doing pilot studies to test the usability of the system for teaching chiropractic and physical therapy procedures." added Greg Kawchuk, a co-supervisor on the project from the Faculty of Rehabilitation Medicine. Once these pilot studies are complete, the research team expects the deployment of the system in real surgical pilot studies will quickly follow.

Watts is co-supervised by Boulanger, Cisco Chair in Healthcare Solutions and professor in the Faculty of Science, and by Kawchuk, professor in the Faculty of Rehabilitation Medicine.

ProjectDR was presented last November at the Virtual Reality Software and Technology Symposium in Gothenburg, Sweden.

Most Popular Now

Generative AI's Diagnostic Capabili…

The use of generative AI for diagnostics has attracted attention in the medical field and many research papers have been published on this topic. However, because the evaluation criteria were...

AI Tool can Track Effectiveness of Multi…

A new artificial intelligence (AI) tool that can help interpret and assess how well treatments are working for patients with multiple sclerosis (MS) has been developed by UCL researchers. AI uses...

New System for the Early Detection of Au…

A team from the Human-Tech Institute-Universitat Politècnica de València has developed a new system for the early detection of Autism Spectrum Disorder (ASD) using virtual reality and artificial intelligence. The...

Diagnoses and Treatment Recommendations …

A new study led by Prof. Dan Zeltzer, a digital health expert from the Berglas School of Economics at Tel Aviv University, compared the quality of diagnostic and treatment recommendations...

Surrey and Sussex Healthcare NHS Trust g…

Surrey and Sussex Healthcare NHS Trust has marked an important milestone in connecting busy radiologists across large parts of South East England, following the successful go live of Sectra's enterprise...

AI Tool Helps Predict Relapse of Pediatr…

Artificial intelligence (AI) shows tremendous promise for analyzing vast medical imaging datasets and identifying patterns that may be missed by human observers. AI-assisted interpretation of brain scans may help improve...

Detecting Lung Cancer 4 Months Earlier a…

GPs may soon be able to identify patients with an increased risk of lung cancer up to 4 months earlier than is currently the case. The GP should be able...

An AI Tool Grounded in Evidence-Based Me…

A powerful clinical artificial intelligence tool developed by University at Buffalo biomedical informatics researchers has demonstrated remarkable accuracy on all three parts of the United States Medical Licensing Exam (Step...

Infectious Disease Surveillance Platform…

The Biothreats Emergence, Analysis and Communications Network (BEACON) leverages advanced artificial intelligence (AI), large language models (LLMs) and a network of globally based experts to rapidly collect, analyze, and disseminate...

Right Patient, Right Dose, Right Time

While artificial intelligence (AI) has shown promising potential, much of its use has remained theoretical or retrospective. Turning its potential into real-world healthcare outcomes, researchers at the Yong Loo Lin...

AI-Powered Analysis of Stent Healing

Each year, more than three million people worldwide are treated with stents to open blocked blood vessels caused by heart disease. However, monitoring the healing process after implantation remains a...

Children's Health Ireland to Transf…

Healthcare teams responsible for paediatric care in Ireland are to save significant time in accessing important diagnostic imaging and reports, with the help of a new agreement with medical imaging...