Yet the researchers caution that AI isn't a replacement for human expertise and that their tool shouldn’t be used as such.
"If you use the tool smartly, it can help you," said Lubomir Hadjiyski, Ph.D., a professor of radiology at the University of Michigan Medical School and the senior author of the study.
When patients develop bladder cancer, surgeons often remove the entire bladder in an effort to keep the cancer from returning or spreading to other organs or areas. More evidence is building, though, that surgery may not be necessary if a patient has zero evidence of disease after chemotherapy.
However, it's difficult to determine whether the lesion left after treatment is simply tissue that’s become necrotic or scarred as a result of treatment or whether cancer remains. The researchers wondered if AI could help.
"The big question was when you have such an artificial device next to you, how is it going to affect the physician?" Hadjiyski said. "Is it going to help? Is it going to confuse them? Is it going to raise their performance or will they simply ignore it?"
Fourteen physicians from different specialties - including radiology, urology and oncology - as well as two fellows and a medical student looked at pre- and post-treatment scans of 157 bladder tumors. The providers gave ratings for three measures that assessed the level of response to chemotherapy as well as a recommendation for the next treatment to be done for each patient (radiation or surgery).
Then the providers looked at a score calculated by the computer. Lower scores indicated a lower likelihood of complete response to chemo and vice versa for higher scores. The providers could revise their ratings or leave them unchanged. Their final ratings were compared against samples of the tumors taken during their bladder removal surgeries to gauge accuracy.
Across different specialties and experience levels, providers saw improvements in their assessments with the AI system. Those with less experience had even more gains, so much so that they were able to make diagnoses at the same level as the experienced participants.
"That was the distinct part of that study that showed interesting observations about the audience," Hadjiyski said.
The tool helped providers from academic institutions more than those that worked at health centers focused solely on clinical care.
The study is part of an NIH-funded project, led by Hadjiyski and Ajjai Alva, M.D., an associate professor of internal medicine at U-M, to develop and evaluate biomarker-based tools for treatment response decision support of bladder cancer.
Over the course of more than two decades of conducting AI-based studies to assess different types of cancer and their treatment response, Hadjiyski says he’s observed that machine learning tools can be useful as a second opinion to assist physicians in decision making, but they can also make mistakes.
"One interesting thing that we figured out is that the computer makes mistakes on a different subset of cases than a radiologist would," he added. "Which means that if the tool is used correctly, it gives a chance to improve but not replace the physician's judgment."
Sun D, Hadjiiski L, Alva A, Zakharia Y, Joshi M, Chan HP, Garje R, Pomerantz L, Elhag D, Cohan RH, Caoili EM, Kerr WT, Cha KH, Kirova-Nedyalkova G, Davenport MS, Shankar PR, Francis IR, Shampain K, Meyer N, Barkmeier D, Woolen S, Palmbos PL, Weizer AZ, Samala RK, Zhou C, Matuszak M.
Computerized Decision Support for Bladder Cancer Treatment Response Assessment in CT Urography: Effect on Diagnostic Accuracy in Multi-Institution Multi-Specialty Study.
Tomography. 2022 Mar 2;8(2):644-656. doi: 10.3390/tomography8020054