Abstract
The evaluation of explanation methods has become a significant issue in explainable artificial intelligence (XAI) due to the recent surge of opaque AI models in decision support systems (DSS). Explanations are essential for bias detection and control of uncertainty since most accurate AI models are opaque with low transparency and comprehensibility. There are numerous criteria to choose from when evaluating explanation method quality. However, since existing criteria focus on evaluating single explanation methods, it is not obvious how to compare the quality of different methods.
In this paper, we have conducted a semi-systematic meta-survey over fifteen literature surveys covering the evaluation of explainability to identify existing criteria usable for comparative evaluations of explanation methods.
The main contribution in the paper is the suggestion to use appropriate trust as a criterion to measure the outcome of the subjective evaluation criteria and consequently make comparative evaluations possible. We also present a model of explanation quality aspects. In the model, criteria with similar definitions are grouped and related to three identified aspects of quality; model, explanation, and user. We also notice four commonly accepted criteria (groups) in the literature, covering all aspects of explanation quality: Performance, appropriate trust, explanation satisfaction, and fidelity. We suggest the model be used as a chart for comparative evaluations to create more generalisable research in explanation quality.
This research is partly founded by the Swedish Knowledge Foundation through the Industrial Research School INSiDR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Murdoch, W.J., Singh, C., Kumbier, K., Abbasi-Asl, R., Yu, B.: Definitions, methods, and applications in interpretable machine learning. Proc. Natl. Acad. Sci. 116(44), 22071–22080 (2019)
Snyder, H.: Literature review as a research methodology: an overview and guidelines. J. Bus. Res. 104, 333–339 (2019)
Webster, J., Watson, R.T.: Analyzing the past to prepare for the future: writing a literature review. MIS Q. xiii–xxiii (2002)
Löfström, H., Hammar, K., Johansson, U.: A meta survey of quality evaluation criteria in explanation methods (2022)
Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 4768–4777 (2017)
Moradi, M., Samwald, M.: Post-hoc explanation of black-box classifiers using confident itemsets. Expert Syst. Appl. 165, 113941 (2021)
Arrieta, A.B., et al.: Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020)
Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)
Gilpin, L.H., Bau, D., Yuan, B.Z., Bajwa, A., Specter, M., Kagal, L.: Explaining explanations: an overview of interpretability of machine learning. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 80–89. IEEE (2018)
Hoffman, R.R., Mueller, S.T., Klein, G., Litman, J.: Metrics for explainable AI: challenges and prospects. arXiv preprint arXiv:1812.04608 (2018)
Carvalho, D.V., Pereira, E.M., Cardoso, J.S.: Machine learning interpretability: a survey on methods and metrics. Electronics 8, 832 (2019)
Mueller, S.T., Hoffman, R.R., Clancey, W., Emrey, A., Klein, G.: Explanation in human-AI systems: a literature meta-review, synopsis of key ideas and publications, and bibliography for explainable AI. arXiv preprint arXiv:1902.01876 (2019)
Mohseni, S., Zarei, N., Ragan, E.D.: A multidisciplinary survey and framework for design and evaluation of explainable AI systems. arXiv, pp. arXiv-1811 (2018)
Gunning, D., Aha, D.W.: Darpa’s explainable artificial intelligence program. AI Mag 40(2), 44–58 (2019)
Hoff, K.A., Bashir, M.: Trust in automation: integrating empirical evidence on factors that influence trust. Hum. Factors 57(3), 407–434 (2015)
Zhou, J., Gandomi, A.H., Chen, F., Holzinger, A.: Evaluating the quality of machine learning explanations: a survey on methods and metrics. Electronics 10(5), 593 (2021)
Dzindolet, M.T., Peterson, S.A., Pomranky, R.A., Pierce, L.G., Beck, H.P.: The role of trust in automation reliance. Int. J. Hum Comput Stud. 58(6), 697–718 (2003)
Pavlidis, M., Mouratidis, H., Islam, S., Kearney, P.: Dealing with trust and control: a meta-model for trustworthy information systems development. In: 2012 Sixth International Conference on Research Challenges in Information Science (RCIS), pp. 1–9. IEEE (2012)
Yang, F., Huang, Z., Scholtz, J., Arendt, D.L.: How do visual explanations foster end users’ appropriate trust in machine learning? In: Proceedings of the 25th International Conference on Intelligent User Interfaces, pp. 189–201 (2020)
Marsh, S., Dibben, M.R.: Trust, untrust, distrust and mistrust – an exploration of the dark(er) side. In: Herrmann, P., Issarny, V., Shiu, S. (eds.) iTrust 2005. LNCS, vol. 3477, pp. 17–33. Springer, Heidelberg (2005). https://doi.org/10.1007/11429760_2
Ekman, F., Johansson, M., Sochor, J.: Creating appropriate trust in automated vehicle systems: a framework for HMI design. IEEE Trans. Hum. Mach. Syst. 48(1), 95–101 (2017)
McDermott, P.L., Ten Brink, R.N.: Practical guidance for evaluating calibrated trust. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 63, pp. 362–366. SAGE Publications Sage CA, Los Angeles (2019)
Chromik, M., Schuessler, M.: A taxonomy for human subject evaluation of black-box explanations in xai. In ExSS-ATEC@ IUI (2020)
Das, A., Rad, P.: Opportunities and challenges in explainable artificial intelligence (XAI): a survey. arXiv preprint arXiv:2006.11371 (2020)
Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)
Wang, D., Yang, Q., Abdul, A., Lim, B.Y.: Designing theory-driven user-centric explainable AI. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, pp. 1–15, New York, Association for Computing Machinery (2019)
Zhang, Y., Chen, X.: Explainable recommendation: a survey and new perspectives. arXiv preprint arXiv:1804.11192 (2018)
Holzinger, A., Carrington, A., Müller, H.: Measuring the quality of explanations: the system causability scale (SCS). KI-Künstliche Intelligenz, pp. 1–6 (2020)
Linardatos, P., Papastefanopoulos, V., Kotsiantis, S.: Explainable AI: a review of machine learning interpretability methods. Entropy 23(1), 18 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Löfström, H., Hammar, K., Johansson, U. (2022). A Meta Survey of Quality Evaluation Criteria in Explanation Methods. In: De Weerdt, J., Polyvyanyy, A. (eds) Intelligent Information Systems. CAiSE 2022. Lecture Notes in Business Information Processing, vol 452. Springer, Cham. https://doi.org/10.1007/978-3-031-07481-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-07481-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-07480-6
Online ISBN: 978-3-031-07481-3
eBook Packages: Computer ScienceComputer Science (R0)