Abstract
Reliable uncertainty estimation is one of the major challenges for medical classification tasks. While many approaches have been proposed, recently the statistical framework of conformal predictions has gained a lot of attention, due to its ability to provide provable calibration guarantees. Nonetheless, the application of conformal predictions in safety-critical areas such as medicine comes with pitfalls, limitations and assumptions that practitioners need to be aware of. We demonstrate through examples from dermatology and histopathology that conformal predictions are unreliable under distributional shifts in input and label variables. Additionally, conformal predictions should not be used for selecting predictions to improve accuracy and are not reliable for subsets of the data, such as individual classes or patient attributes. Moreover, in classification settings with a small number of classes, which are common in medical image classification tasks, conformal predictions have limited practical value.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Begoli, E., Bhattacharya, T., Kusnezov, D.: The need for uncertainty quantification in machine-assisted medical decision making. Nat. Mach. Intell. 1(1), 20–23 (2019). https://doi.org/10.1038/s42256-018-0004-1
Van der Laak, J., Litjens, G., Ciompi, F.: Deep learning in histopathology: the path to the clinic. Nat. Med. 27(5), 775–784 (2021). https://doi.org/10.1038/s41591-021-01343-4
Kompa, B., Snoek, J., Beam, A.L.: Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digit. Med. 4(1), 4 (2021). https://doi.org/10.1038/S41746-020-00367-3
Jaeger, P.F., Lüth, C.T., Klein, L., Bungert, T.J.: A call to reflect on evaluation practices for failure detection in image classification. arXiv preprint arXiv:2211.15259 (2022). https://doi.org/10.48550/arXiv.2211.15259
Mehrtens, H.A., Kurz, A., Bucher, T.-C., Brinker, T.J.: Benchmarking common uncertainty estimation methods with histopathological images under domain shift and label noise. Med. Image Anal. (2023). https://doi.org/10.1016/j.media.2023.102914
Vovk, V., Gammerman, A., Shafer, G.: Algorithmic Learning in a Random World. Springer, New York (2005). https://doi.org/10.1007/b106715
Wieslander, H., et al.: Deep learning with conformal prediction for hierarchical analysis of large-scale whole-slide tissue images. IEEE J. Biomed. Health Inform. 25(2), 371–380 (2020). https://doi.org/10.1109/JBHI.2020.2996300
Olsson, H., et al.: Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction. Nat. Commun. 13(1), 7761 (2022). https://doi.org/10.1038/s41467-022-34945-8
Lu, C., Lemay, A., Chang, K., Höbel, K., Kalpathy-Cramer, J.: Fair conformal predictors for applications in medical imaging. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 12008–12016 (2022). https://doi.org/10.1609/aaai.v36i11.21459
Lu, C., Chang, K., Singh, P., Kalpathy-Cramer, J.: Three applications of conformal prediction for rating breast density in mammography. arXiv preprint arXiv:2206.12008 (2022). https://doi.org/10.48550/ARXIV.2206.12008
Lu, C., Angelopoulos, A.N., Pomerantz, S.: Improving trustworthiness of AI disease severity rating in medical imaging with ordinal conformal prediction sets. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 545–554 (2022). https://doi.org/10.48550/ARXIV.2207.02238
Angelopoulos, A.N., Bates, S.: A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 (2021)
Geifman, Y., El-Yaniv, R.: Selective classification for deep neural networks. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Romano, Y., Sesia, M., Candes, E.: Classification with valid and adaptive coverage. Adv. Neural. Inf. Process. Syst. 33, 3581–3591 (2020). https://doi.org/10.48550/arXiv.2006.02544
Angelopoulos, A., Bates, S., Malik, J., Jordan, M.I.: Uncertainty sets for image classifiers using conformal prediction. arXiv preprint arXiv:2009.14193 (2020). https://doi.org/10.48550/arXiv.2009.14193
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5(1), 1–9 (2018). https://doi.org/10.1038/sdata.2018.161
Bandi, P., et al.: From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge. IEEE Trans. Med. Imaging 38(2), 550–560 (2018). https://doi.org/10.1109/TMI.2018.2867350
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)
Maddox, W.J., Izmailov, P., Garipov, T., Vetrov, D.P., Wilson, A.G.: A simple baseline for Bayesian uncertainty in deep learning. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Vovk, V.: Conditional validity of inductive conformal predictors. In: Asian Conference on Machine Learning, pp. 475–490 (2012). https://doi.org/10.1007/s10994-013-5355-6
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a largescale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848
Acknowledgements
This publication is funded by the ‘Ministerium für Soziales, Gesundheit und Integration’, Baden Württemberg, Germany, as part of the ‘KI-Translations-Initiative’. Titus Josef Brinker owns a company that develops mobile apps (Smart Health Heidelberg GmbH, Heidelberg, Germany), outside of the scope of the submitted work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mehrtens, H., Bucher, T., Brinker, T.J. (2023). Pitfalls of Conformal Predictions for Medical Image Classification. In: Sudre, C.H., Baumgartner, C.F., Dalca, A., Mehta, R., Qin, C., Wells, W.M. (eds) Uncertainty for Safe Utilization of Machine Learning in Medical Imaging. UNSURE 2023. Lecture Notes in Computer Science, vol 14291. Springer, Cham. https://doi.org/10.1007/978-3-031-44336-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-44336-7_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44335-0
Online ISBN: 978-3-031-44336-7
eBook Packages: Computer ScienceComputer Science (R0)
