Augmented Reality System Lets Doctors See under Patients' Skin without the Scalpel

New technology is bringing the power of augmented reality into clinical practice. The system, called ProjectDR, allows medical images such as CT scans and MRI data to be displayed directly on a patient's body in a way that moves as the patient does.

"We wanted to create a system that would show clinicians a patient's internal anatomy within the context of the body," explained Ian Watts, a computing science graduate student and the developer of ProjectDR.

The technology includes a motion-tracking system using infrared cameras and markers on the patient's body, as well as a projector to display the images. But the really difficult part, Watts explained, is having the image track properly on the patient's body even as they shift and move. The solution: custom software written by Watts that gets all of the components working together.

Vast applications

"There are lots of applications for this technology, including in teaching, physiotherapy, laparoscopic surgery and even surgical planning," said Watts, who developed the technology with fellow graduate student Michael Fiest.

ProjectDR also has the capacity to present segmented images - for example, only the lungs or only the blood vessels - depending on what a clinician is interested in seeing.

For now, Watts is working on refining ProjectDR to improve the system's automatic calibration and to add components such as depth sensors. The next steps are testing the program's viability in a clinical setting, explained Pierre Boulanger, professor in the Department of Computing Science.

Next steps

"Soon, we'll deploy ProjectDR in an operating room in a surgical simulation laboratory to test the pros and cons in real-life surgical applications," said Boulanger. "We are also doing pilot studies to test the usability of the system for teaching chiropractic and physical therapy procedures." added Greg Kawchuk, a co-supervisor on the project from the Faculty of Rehabilitation Medicine. Once these pilot studies are complete, the research team expects the deployment of the system in real surgical pilot studies will quickly follow.

Watts is co-supervised by Boulanger, Cisco Chair in Healthcare Solutions and professor in the Faculty of Science, and by Kawchuk, professor in the Faculty of Rehabilitation Medicine.

ProjectDR was presented last November at the Virtual Reality Software and Technology Symposium in Gothenburg, Sweden.

Most Popular Now

Unlocking the 10 Year Health Plan

The government's plan for the NHS is a huge document. Jane Stephenson, chief executive of SPARK TSL, argues the key to unlocking its digital ambitions is to consider what it...

Alcidion Grows Top Talent in the UK, wit…

Alcidion has today announced the addition of three new appointments to their UK-based team, with one internal promotion and two external recruits. Dr Paul Deffley has been announced as the...

AI can Find Cancer Pathologists Miss

Men assessed as healthy after a pathologist analyses their tissue sample may still have an early form of prostate cancer. Using AI, researchers at Uppsala University have been able to...

AI, Full Automation could Expand Artific…

Automated insulin delivery (AID) systems such as the UVA Health-developed artificial pancreas could help more type 1 diabetes patients if the devices become fully automated, according to a new review...

New Training Year Starts at Siemens Heal…

In September, 197 school graduates will start their vocational training or dual studies in Germany at Siemens Healthineers. 117 apprentices and 80 dual students will begin their careers at Siemens...

How AI could Speed the Development of RN…

Using artificial intelligence (AI), MIT researchers have come up with a new way to design nanoparticles that can more efficiently deliver RNA vaccines and other types of RNA therapies. After training...

MIT Researchers Use Generative AI to Des…

With help from artificial intelligence, MIT researchers have designed novel antibiotics that can combat two hard-to-treat infections: drug-resistant Neisseria gonorrhoeae and multi-drug-resistant Staphylococcus aureus (MRSA). Using generative AI algorithms, the research...

AI Hybrid Strategy Improves Mammogram In…

A hybrid reading strategy for screening mammography, developed by Dutch researchers and deployed retrospectively to more than 40,000 exams, reduced radiologist workload by 38% without changing recall or cancer detection...

Penn Developed AI Tools and Datasets Hel…

Doctors treating kidney disease have long depended on trial-and-error to find the best therapies for individual patients. Now, new artificial intelligence (AI) tools developed by researchers in the Perelman School...

Are You Eligible for a Clinical Trial? C…

A new study in the academic journal Machine Learning: Health discovers that ChatGPT can accelerate patient screening for clinical trials, showing promise in reducing delays and improving trial success rates. Researchers...

Global Study Reveals How Patients View M…

How physicians feel about artificial intelligence (AI) in medicine has been studied many times. But what do patients think? A team led by researchers at the Technical University of Munich...

New AI Tool Addresses Accuracy and Fairn…

A team of researchers at the Icahn School of Medicine at Mount Sinai has developed a new method to identify and reduce biases in datasets used to train machine-learning algorithms...