Effect of computerized decision support on diagnostic accuracy and intra-observer variability in multi-institutional observer performance study for bladder cancer treatment response assessment in CT urography

Di Sun; Lubomir Hadjiiski; Rohan Garje; Yousef Zakharia; Lauren Pomerantz; Monika Joshi; Ajjai Alva; Heang-Ping Chan; Richard H. Cohan M.D.; Elaine M. Caoili M.D.; Kenny H. Cha; Galina Kirova-Nedyalkova; Matthew S. Davenport; Prasad R. Shankar; Isaac R. Francis; Kimberly Shampain; Nathaniel Meyer; Daniel Barkmeier; Sean Woolen; Phillip L. Palmbos; Alon Z. Weizer; Ravi K. Samala; Chuan Zhou; Martha Matuszak

doi:10.1117/12.2611179

4 April 2022 Effect of computerized decision support on diagnostic accuracy and intra-observer variability in multi-institutional observer performance study for bladder cancer treatment response assessment in CT urography

Di Sun, Lubomir Hadjiiski, Rohan Garje, Yousef Zakharia, Lauren Pomerantz, Monika Joshi, Ajjai Alva, Heang-Ping Chan, Richard H. Cohan M.D., Elaine M. Caoili M.D., Kenny H. Cha, Galina Kirova-Nedyalkova, Matthew S. Davenport, Prasad R. Shankar, Isaac R. Francis, Kimberly Shampain, Nathaniel Meyer, Daniel Barkmeier, Sean Woolen, Phillip L. Palmbos, Alon Z. Weizer, Ravi K. Samala, Chuan Zhou, Martha Matuszak

Author Affiliations +

Proceedings Volume 12033, Medical Imaging 2022: Computer-Aided Diagnosis; 120330G (2022) https://doi.org/10.1117/12.2611179
Event: SPIE Medical Imaging, 2022, San Diego, California, United States

Abstract

We have previously developed a computerized decision support system for bladder cancer treatment response assessment (CDSS-T) in CT urography (CTU). In this work, we conducted an observer study to evaluate the diagnostic accuracy and intra-observer variability with and without the CDSS-T system. One hundred fifty-seven pre- and posttreatment lesion pairs were identified in pre- and post- chemotherapy CTU scans of 123 patients. Forty lesion pairs had T0 stage (complete response) after chemotherapy. Multi-disciplinary observers from 4 different institutions participated in reading the lesion pairs, including 5 abdominal radiologists, 4 radiology residents, 5 oncologists, 1 urologist, and 1 medical student. Each observer provided estimates of the T0 likelihood after treatment without and then with the CDSST aid for each lesion. To assess the intra-observer variability, 51 cases were evaluated two times – the original and the repeated evaluation. The average area under the curve (AUC) of 16 observers for estimation of T0 disease after treatment increased from 0.73 without CDSS-T to 0.77 with CDSS-T (p = 0.003). For the evaluation with CDSS-T, the average AUC performance for different institutions was similar. The performance with CDSS-T was improved significantly and the AUC standard deviations were slightly smaller showing potential trend of more accurate and uniform performance with CDSS-T. There was no significant difference between the original and repeated evaluation. This study demonstrated that our CDSS-T system has the potential to improve treatment response assessment of physicians from different specialties and institutions, and reduce the inter- and intra-observer variabilities of the assessments.

Conference Presentation

Citation Download Citation

Di Sun, Lubomir Hadjiiski, Rohan Garje, Yousef Zakharia, Lauren Pomerantz, Monika Joshi, Ajjai Alva, Heang-Ping Chan, Richard H. Cohan M.D., Elaine M. Caoili M.D., Kenny H. Cha, Galina Kirova-Nedyalkova, Matthew S. Davenport, Prasad R. Shankar, Isaac R. Francis, Kimberly Shampain, Nathaniel Meyer, Daniel Barkmeier, Sean Woolen, Phillip L. Palmbos, Alon Z. Weizer, Ravi K. Samala, Chuan Zhou, and Martha Matuszak "Effect of computerized decision support on diagnostic accuracy and intra-observer variability in multi-institutional observer performance study for bladder cancer treatment response assessment in CT urography", Proc. SPIE 12033, Medical Imaging 2022: Computer-Aided Diagnosis, 120330G (4 April 2022); https://doi.org/10.1117/12.2611179

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available