Abstract
Trust is an important prerequisite for the acceptance of an Artificial Intelligence (AI) system, in particular in the medical domain. Explainability is currently discussed as the key approach to induce trust. Since a medical AI system is considered a medical device, it also has to be formally certified by an officially recognised agency. The paper argues that neither explainability nor certification suffice to tackle the trust problem. Instead, we propose an alternative approach aimed at showing the physician how well a patient is represented in the original training data set. We operationalize this approach by developing formal indicators and illustrate their usefulness with a real-world medical data set.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chien, S.-Y., Lewis, M., Semnani-Azad, Z., Sycara, K.: An empirical model of cultural factors on trust in automation. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 58, pp. 859–863, October 2014
Choi, E., Bahadori, M.T., Schuetz, A., Stewart, W.F., Sun, J.: Doctor AI: predicting clinical events via recurrent neural networks. In: Machine Learning for Healthcare Conference, pp. 301–318 (2016)
Došilović, F.K., Brčić, M., Hlupić, N.: Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 0210–0215. IEEE (2018)
Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: Proceedings of 2018 IEEE 5th International Conference on Data Science and Advanced Analytics, pp. 80–89, October 2018
Korot, E., et al.: Will AI replace ophthalmologists? Transl. Vis. Sci. Technol. 9(2), 1–5 (2020)
Leonard, J.A., Kramer, M.A., Ungar, L.H.: A neural network architecture that computes its own reliability. Comput. Chem. Eng. 16(9), 819–835 (1992)
Mandal, S., Greenblatt, A.B., An, J.: Imaging intelligence: AI is transforming medical imaging across the imaging spectrum. IEEE Pulse 9(5), 16–24 (2018)
Matheny, M., Israni, S.T., Ahmed, M., Whicher, D.: Artificial intelligence in health care: the hope, the hype, the promise, the peril. Natl. Acad. Med. 94–97 (2020)
Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1–38 (2019)
Reimer, U., Maier, E., Tödtli, B.: Going beyond explainability in medical AI systems. In: Proceedings of Modellierung 2020 Short Papers, Workshop Papers, and Tools & Demo Papers, vol. 2542, pp. 185–191. CEUR-WS.org (2020)
Ribeiro, M.T., Singh, S., Guestrin, C.: why should i trust you?: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)
Ronicke, S., Hirsch, M.C., Türk, E., Larionov, K., Tientcheu, D., Wagner, A.D.: Thu0564 could a probabilistic reasoning AI accelerate rare disease diagnosis? Evaluating the potential impact of a diagnostic decision support system in a retrospective study. Ann. Rheum. Dis. 78(Suppl 2), 572–574 (2019)
Savadjiev, P., et al.: Demystification of AI-driven medical image interpretation: past, present and future. Eur. Radiol. 29(3), 1616–1624 (2019)
Schulam, P., Saria, S.: Can you trust this prediction? Auditing pointwise reliability after learning. In: Proceedings of 22nd International Conference on Artificial Intelligence and Statistics, pp. 1022–1031 (2019)
Topol, E.J.: High-performance medicine: the convergence of human and artificial intelligence. Nat. Med. 25(1), 44–56 (2019)
Ulianova, S.: Cardiovascular disease dataset. https://www.kaggle.com/sulianova/cardiovascular-disease-dataset. Accessed 17 Oct 2019
Zafar, M.B., Valera, I., Gomez-Rodriguez, M., Gummadi, K.P.: Fairness constraints: a flexible approach for fair classification. J. Mach. Learn. Res. 20(75), 1–42 (2019)
Zhou, N., et al.: Concordance study between IBM watson for oncology and clinical practice for patients with cancer in China. Oncologist 24(6), 812 (2019)
Acknowledgement
We would like to thank the reviewers for their helpful comments to an earlier version of the paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Reimer, U., Tödtli, B., Maier, E. (2020). How to Induce Trust in Medical AI Systems. In: Grossmann, G., Ram, S. (eds) Advances in Conceptual Modeling. ER 2020. Lecture Notes in Computer Science(), vol 12584. Springer, Cham. https://doi.org/10.1007/978-3-030-65847-2_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-65847-2_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65846-5
Online ISBN: 978-3-030-65847-2
eBook Packages: Computer ScienceComputer Science (R0)