Chunxu Yang
ABSTRACT
This demo presents a human-AI collaborative system to assist pathologists in examining the pathological pattern of mitosis, a critical factor in tumor diagnosis. Traditionally, pathologists face challenges in assessing mitoses due to the task’s inherent complexity. The demonstrated system aims to address the problem by designing an enhanced human-artificial intelligence workflow. Firstly, it can guide a pathologist user to regions of interest that have flexible morphologies. Then, inside each region of interest, the system highlights each AI-detected mitosis event along with enriched forms of explainable AI evidence. The system can potentially improve the efficiency and correctness of pathologists’ mitosis assessment by enabling them to leverage the power of AI while retaining their clinical expertise and judgment.
Supplemental Material
- Marc Aubreville, Christof A. Bertram, Taryn A. Donovan, Christian Marzahl, Andreas Maier, and Robert Klopfleisch. 2020. A completely annotated whole slide image dataset of canine breast cancer to aid human breast cancer research. Scientific Data 7, 1 (Nov. 2020), 417. https://doi.org/10.1038/s41597-020-00756-zGoogle Scholar
Cross Ref
- Marc Aubreville, Christof A. Bertram, Christian Marzahl, Corinne Gurtner, Martina Dettwiler, Anja Schmidt, Florian Bartenschlager, Sophie Merz, Marco Fragoso, Olivia Kershaw, Robert Klopfleisch, and Andreas Maier. 2020. Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region. Scientific Reports 10, 1 (Oct. 2020), 16447. https://doi.org/10.1038/s41598-020-73246-2Google ScholarCross Ref
- Marc Aubreville, Nikolas Stathonikos, Taryn A Donovan, Robert Klopfleisch, Jonathan Ganz, Jonas Ammeling, Frauke Wilm, Mitko Veta, Samir Jabari, Markus Eckstein, 2023. Domain generalization across tumor types, laboratories, and species–insights from the 2022 edition of the Mitosis Domain Generalization Challenge. arXiv preprint arXiv:2309.15589 (2023).Google Scholar
- Murat Seckin Ayhan and Philipp Berens. 2022. Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In Medical Imaging with Deep Learning.Google Scholar
- Maschenka C.A. Balkenhol, David Tellez, Willem Vreuls, Pieter C. Clahsen, Hans Pinckaers, Francesco Ciompi, Peter Bult, and Jeroen A.W.M. Van Der Laak. 2019. Deep learning assisted mitotic counting for breast cancer. Laboratory Investigation 99, 11 (Nov. 2019), 1596–1606. https://doi.org/10.1038/s41374-019-0275-0Google ScholarCross Ref
- Christof A Bertram, Marc Aubreville, Taryn A Donovan, Alexander Bartel, Frauke Wilm, Christian Marzahl, Charles-Antoine Assenmacher, Kathrin Becker, Mark Bennett, Sarah Corner, 2022. Computer-assisted mitotic count using a deep learning–based algorithm improves interobserver reproducibility and accuracy. Veterinary Pathology 59, 2 (March 2022), 211–226. https://doi.org/10.1177/03009858211067478Google ScholarCross Ref
- Christof A. Bertram, Marc Aubreville, Corinne Gurtner, Alexander Bartel, Sarah M. Corner, Martina Dettwiler, Olivia Kershaw, Erica L. Noland, Anja Schmidt, Dodd G. Sledge, Rebecca C. Smedley, Tuddow Thaiwong, Matti Kiupel, Andreas Maier, and Robert Klopfleisch. 2020. Computerized Calculation of Mitotic Count Distribution in Canine Cutaneous Mast Cell Tumor Sections: Mitotic Count Is Area Dependent. Veterinary Pathology 57, 2 (2020), 214–226. https://doi.org/10.1177/0300985819890686 arXiv:https://doi.org/10.1177/0300985819890686PMID: 31808382.Google ScholarCross Ref
- Carrie J. Cai, Emily Reif, Narayan Hegde, Jason Hipp, Been Kim, Daniel Smilkov, Martin Wattenberg, Fernanda Viegas, Greg S. Corrado, Martin C. Stumpe, and Michael Terry. 2019. Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300234Google ScholarDigital Library
- Francisco Maria Calisto, João Fernandes, Margarida Morais, Carlos Santiago, João Maria Abrantes, Nuno Nunes, and Jacinto C. Nascimento. 2023. Assertiveness-based Agent Communication for a Personalized Medicine on Medical Imaging Diagnosis. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 13, 20 pages. https://doi.org/10.1145/3544548.3580682Google ScholarDigital Library
- Francisco Maria Calisto, Carlos Santiago, Nuno Nunes, and Jacinto C. Nascimento. 2022. BreastScreening-AI: Evaluating medical intelligent agents for human-AI interactions. Artificial Intelligence in Medicine 127 (2022), 102285. https://doi.org/10.1016/j.artmed.2022.102285Google ScholarDigital Library
- Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. 2018. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). 839–847. https://doi.org/10.1109/WACV.2018.00097Google ScholarCross Ref
- Ian A. Cree, Puay Hoon Tan, William D. Travis, Pieter Wesseling, Yukako Yagi, Valerie A. White, Dilani Lokuhetty, and Richard A. Scolyer. 2021. Counting mitoses: SI(ze) matters!Modern Pathology 34, 9 (Sept. 2021), 1651–1657. https://doi.org/10.1038/s41379-021-00825-7Google ScholarCross Ref
- Theodore Evans, Carl Orge Retzlaff, Christian Geißler, Michaela Kargl, Markus Plass, Heimo Müller, Tim-Rasmus Kiehl, Norman Zerbe, and Andreas Holzinger. 2022. The explainability paradox: Challenges for xAI in digital pathology. Future Generation Computer Systems 133 (2022), 281–296. https://doi.org/10.1016/j.future.2022.03.009Google ScholarDigital Library
- Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning. PMLR, 1050–1059.Google Scholar
- Roland Goldbrunner, Pantelis Stavrinou, Michael D Jenkinson, Felix Sahm, Christian Mawrin, Damien C Weber, Matthias Preusser, Giuseppe Minniti, Morten Lund-Johansen, Florence Lefranc, Emanuel Houdart, Kita Sallabanda, Emilie Le Rhun, David Nieuwenhuizen, Ghazaleh Tabatabai, Riccardo Soffietti, and Michael Weller. 2021. EANO guideline on the diagnosis and management of meningiomas. Neuro-Oncology 23, 11 (06 2021), 1821–1834. https://doi.org/10.1093/neuonc/noab150 arXiv:https://academic.oup.com/neuro-oncology/article-pdf/23/11/1821/41063953/noab150.pdfGoogle ScholarCross Ref
- Hongyan Gu, Mohammad Haeri, Shuo Ni, Christopher Kazu Williams, Neda Zarrin-Khameh, Shino Magaki, and Xiang ‘Anthony’ Chen. 2023. Detecting Mitoses with a Convolutional Neural Network for MIDOG 2022 Challenge. In Mitosis Domain Generalization and Diabetic Retinopathy Analysis, Bin Sheng and Marc Aubreville (Eds.). Springer Nature Switzerland, Cham, 211–216.Google Scholar
- Hongyan Gu, Jingbin Huang, Lauren Hung, and Xiang ’Anthony’ Chen. 2021. Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 10 (apr 2021), 25 pages. https://doi.org/10.1145/3449084Google ScholarDigital Library
- Hongyan Gu, Yuan Liang, Yifan Xu, Christopher Kazu Williams, Shino Magaki, Negar Khanlou, Harry Vinters, Zesheng Chen, Shuo Ni, Chunxu Yang, Wenzhong Yan, Xinhai Robert Zhang, Yang Li, Mohammad Haeri, and Xiang ‘Anthony’ Chen. 2023. Improving Workflow Integration with XPath: Design and Evaluation of a Human-AI Diagnosis System in Pathology. ACM Trans. Comput.-Hum. Interact. 30, 2, Article 28 (mar 2023), 37 pages. https://doi.org/10.1145/3577011Google ScholarDigital Library
- Hongyan Gu, Chunxu Yang, Issa Al-kharouf, Shino Magaki, Nelli Lakis, Christopher Kazu Williams, Sallam Mohammad Alrosan, Ellie Kate Onstott, Wenzhong Yan, Negar Khanlou, Inma Cobos, Xinhai Robert Zhang, Neda Zarrin-Khameh, Harry V. Vinters, Xiang Anthony Chen, and Mohammad Haeri. 2024. Enhancing mitosis quantification and detection in meningiomas with computational digital pathology. Acta Neuropathologica Communications 12, 1 (Jan. 2024), 7. https://doi.org/10.1186/s40478-023-01707-6Google ScholarCross Ref
- Hongyan Gu, Chunxu Yang, Mohammad Haeri, Jing Wang, Shirley Tang, Wenzhong Yan, Shujin He, Christopher Kazu Williams, Shino Magaki, and Xiang ’Anthony’ Chen. 2023. Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation System. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 349, 19 pages. https://doi.org/10.1145/3544548.3580694Google ScholarDigital Library
- David N Louis, Arie Perry, Pieter Wesseling, Daniel J Brat, Ian A Cree, Dominique Figarella-Branger, Cynthia Hawkins, H K Ng, Stefan M Pfister, Guido Reifenberger, Riccardo Soffietti, Andreas von Deimling, and David W Ellison. 2021. The 2021 WHO Classification of Tumors of the Central Nervous System: a summary. Neuro-Oncology 23, 8 (06 2021), 1231–1251. https://doi.org/10.1093/neuonc/noab106Google ScholarCross Ref
- Rebecca Randell, Thilina Ambepitiya, Claudia Mello-Thoms, Roy A. Ruddle, David Brettle, Rhys G. Thomas, and Darren Treanor. 2015. Effect of Display Resolution on Time to Diagnosis with Virtual Pathology Slides in a Systematic Search Task. Journal of Digital Imaging 28, 1 (Feb. 2015), 68–76. https://doi.org/10.1007/s10278-014-9726-8Google ScholarCross Ref
- Roy A. Ruddle, Rhys G. Thomas, Rebecca Randell, Philip Quirke, and Darren Treanor. 2016. The Design and Evaluation of Interfaces for Navigating Gigapixel Images in Digital Pathology. ACM Trans. Comput.-Hum. Interact. 23, 1, Article 5 (jan 2016), 29 pages. https://doi.org/10.1145/2834117Google ScholarDigital Library
- Stijn A. Van Bergeijk, Nikolas Stathonikos, Natalie D. Ter Hoeve, Maxime W. Lafarge, Tri Q. Nguyen, Paul J. Van Diest, and Mitko Veta. 2023. Deep learning supported mitoses counting on whole slide images: A pilot study for validating breast cancer grading in the clinical workflow. Journal of Pathology Informatics 14 (2023), 100316. https://doi.org/10.1016/j.jpi.2023.100316Google ScholarCross Ref
- Mitko Veta, Paul J. Van Diest, Mehdi Jiwa, Shaimaa Al-Janabi, and Josien P. W. Pluim. 2016. Mitosis Counting in Breast Cancer: Object-Level Interobserver Agreement and Comparison to an Automatic Method. PLOS ONE 11, 8 (Aug. 2016), e0161286. https://doi.org/10.1371/journal.pone.0161286Google ScholarCross Ref
Index Terms
- A Human-AI Collaborative System to Support Mitosis Assessment in Pathology
Recommendations
Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists
CSCWDespite the promises of data-driven artificial intelligence (AI), little is known about how we can bridge the gulf between traditional physician-driven diagnosis and a plausible future of medicine automated by AI. Specifically, how can we involve AI ...
Augmenting Pathologists with NaviPath: Design and Evaluation of a Human-AI Collaborative Navigation System
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsArtificial Intelligence (AI) brings advancements to support pathologists in navigating high-resolution tumor images to search for pathology patterns of interest. However, existing AI-assisted tools have not realized this promised potential due to a lack ...
Improving Workflow Integration with xPath: Design and Evaluation of a Human-AI Diagnosis System in Pathology
Recent developments in AI have provided assisting tools to support pathologists’ diagnoses. However, it remains challenging to incorporate such tools into pathologists’ practice; one main concern is AI’s insufficient workflow integration with medical ...
Comments