Abstract
Clinical AI applications, particularly medical imaging, are increasingly being adopted in healthcare systems worldwide. However, a crucial question remains: what happens after the AI model is put into production? We present our novel multi-modal model drift framework capable of tracking drift without contemporaneous ground truth using only readily available inputs, namely DICOM metadata, image appearance representation from a variational autoencoder (VAE), and model output probabilities. CheXStray was developed and tested using CheXpert, PadChest and Pediatric Pneumonia Chest X-ray datasets and we demonstrate that our framework generates a strong proxy for ground truth performance. In this work, we offer new insights into the challenges and solutions for observing deployed medical imaging AI and make three key contributions to real-time medical imaging AI monitoring: (1) proof-of-concept for medical imaging drift detection including use of VAE and domain specific statistical methods (2) a multi-modal methodology for measuring and unifying drift metrics (3) new insights into the challenges and solutions for observing deployed medical imaging AI. Our framework is released as open-source tools so that others may easily run their own workflows and build upon our work. Code available at: https://github.com/microsoft/MedImaging-ModelDriftMonitoring
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Benjamens, S., Dhunnoo, P., Meskó, B.: The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit. Med. 3(1), 1–8 (2020)
Bustos, A., Pertusa, A., Salinas, J.M., de la Iglesia-Vayá, M.: PadChest: a large chest x-ray image dataset with multi-label annotated reports. Med. Image Anal. 66, 101797 (2020). https://doi.org/10.1016/j.media.2020.101797
Cao, T., Huang, C., Hui, D.Y.T., Cohen, J.P.: A benchmark of medical out of distribution detection. In: Uncertainty and Robustness in Deep Learning Workshop at ICML (2020), http://arxiv.org/abs/2007.04250
Dikici, E., Bigelow, M., Prevedello, L.M., White, R.D., Erdal, B.S.: Integrating AI into radiology workflow: levels of research, production, and feedback maturity. J. Med. Imag. 7(1), 016502 (2020)
Dodge, Y.: Kolmogorov-Smirnov Test, pp. 283–287. Springer, New York (2008). https://doi.org/10.1007/978-0-387-32833-1_214
Eche, T., Schwartz, L.H., Mokrane, F.Z., Dercle, L.: Toward generalizability in the deployment of artificial intelligence in radiology: role of computation stress testing to overcome underspecification. Radiol. Artif. Intell. 3(6), e210097 (2021)
Finlayson, S.G., et al.: The clinician and dataset shift in artificial intelligence. N. Engl. J. Med. 385(3), 283–286 (2021)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Computer Vision and Pattern Recognition (2017), https://arxiv.org/abs/1608.06993
Irvin, J., et al.: CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. CoRR abs/1901.07031 (2019), http://arxiv.org/abs/1901.07031
Kermany, D., Zhang, K., Goldbaum, M., et al.: Labeled optical coherence tomography (OCT) and chest x-ray images for classification. Mendeley data 2(2) (2018)
Klaise, J., Van Looveren, A., Cox, C., Vacanti, G., Coca, A.: Monitoring and explainability of models in production. arXiv preprint arXiv:2007.06299 (2020)
van Leeuwen, K.G., Schalekamp, S., Rutten, M.J.C.M., van Ginneken, B., de Rooij, M.: Artificial intelligence in radiology: 100 commercially available products and their scientific evidence. Eur. Radiol. 31(6), 3797–3804 (2021). https://doi.org/10.1007/s00330-021-07892-z
Mahajan, V., Venugopal, V.K., Murugavel, M., Mahajan, H.: The algorithmic audit: working with vendors to validate radiology-AI algorithms—how we do it. Acad. Radiol. 27(1), 132–135 (2020)
Mehrizi, M.H.R., van Ooijen, P., Homan, M.: Applications of artificial intelligence (AI) in diagnostic radiology: a technography study. Eur. Radiol. 31(4), 1805–1811 (2021)
Mildenberger, P., Eichelberg, M., Martin, E.: Introduction to the DICOM standard. Eur. Radiol. 12(4), 920–927 (2002)
Pearson, K.: X. on the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Lond. Edinb. Dublin Philos. Mag. J. Sci. 50(302), 157–175 (1900). https://doi.org/10.1080/14786440009463897
Sculley, D., et al.: Machine learning: the high interest credit card of technical debt. In: NIPS Workshop 2014 (2014)
Shafaei, A., Schmidt, M., Little, J.J.: Does your model know the digit 6 is not a cat? a less biased evaluation of “outlier” detectors. CoRR abs/1809.04729 (2018), http://arxiv.org/abs/1809.04729
Tadavarthi, Y., et al.: The state of radiology AI: considerations for purchase decisions and current market offerings. Radiol. Artif. Intell. 2(6), e200004 (2020)
Tariq, A., et al.: Current clinical applications of artificial intelligence in radiology and their best supporting evidence. J. Am. Coll. Radiol. 17(11), 1371–1381 (2020)
West, E., Mutasa, S., Zhu, Z., Ha, R.: Global trend in artificial intelligence-based publications in radiology from 2000 to 2018. Am. J. Roentgenol. 213(6), 1204–1206 (2019)
Wiggins, W.F., et al.: Imaging AI in practice: a demonstration of future workflow using integration standards. Radiol. Artif. Intell. 3(6), e210152 (2021)
Zenati, H., Foo, C.S., Lecouat, B., Manek, G., Chandrasekhar, V.R.: Efficient GAN-based anomaly detection. arXiv preprint arXiv:1802.06222 (2018)
Acknowledgments
This work was was supported in part by the Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI) and Microsoft Health and Life Sciences.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Merkow, J. et al. (2023). CheXstray: A Real-Time Multi-Modal Monitoring Workflow for Medical Imaging AI. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14222. Springer, Cham. https://doi.org/10.1007/978-3-031-43898-1_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-43898-1_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43897-4
Online ISBN: 978-3-031-43898-1
eBook Packages: Computer ScienceComputer Science (R0)
