Skip to main content

Matching the Expert’s Knowledge via a Counterfactual-Based Feature Importance Measure

  • Conference paper
  • First Online:
Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2023)

Abstract

To be employed in real-world applications, explainable artificial intelligence (XAI) techniques need to provide explanations that are comprehensible to experts and decision-makers with no machine learning (ML) background, thus allowing for the validation of the ML model via their domain knowledge.

To this aim, XAI approaches based on feature importance and counterfactuals can be employed, although both have some limitations: the last provide only local explanations, whereas the first can be very computationally expensive. A less computationally-expensive global feature importance measure can be derived by considering the instances close to the model decision boundary and analyzing how often some minor changes in one feature’s values do affect the classification outcome.

However, the validation of XAI techniques in the literature rarely employs the application domain knowledge due to the burden of formalizing it, e.g., providing some degree of expected importance for each feature. Still, given an ML model, it is difficult to determine whether an XAI technique may inject a bias in the explanation (e.g., overestimating or underestimating the importance of a feature) in the absence of such ground truth.

To address this issue, we test our feature importance approach both with the UCI benchmark datasets and real-world smart manufacturing data characterized by annotations provided by domain experts about the expected importance of each feature. If compared to the state-of-the-art, the employed approach results to be reliable and convenient in terms of computation time, as well as more concordant with the expected importance provided by the domain expert.

Work partially supported by (i) the company Koerber Tissue in the project “Data-driven and Artificial Intelligence approaches for Industry 4.0”; and the Italian Ministry of University and Research (MUR) in the frameworks: (ii) PNRR - M4C2 - Investimento 1.3, Partenariato Esteso PE00000013 - “FAIR” - Spoke 1 “Human-centered AI”; (iii) National Center for Sustainable Mobility MOST/Spoke10; and (iv) of the FoReLab project (Departments of Excellence). The authors thank Michelangelo Martorana for his work on the subject during his master’s thesis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)

    Article  Google Scholar 

  2. Afchar, D., Guigue, V., Hennequin, R.: Towards rigorous interpretations: a formalisation of feature attribution. In: International in Proceedings on Machine Learning, pp. 76–86. PMLR (2021)

    Google Scholar 

  3. Ahmed, I., Jeon, G., Piccialli, F.: From artificial intelligence to explainable artificial intelligence in industry 4.0: a survey on what, how, and where. IEEE Trans. Ind. Inf. 18(8), 5031–5042 (2022)

    Article  MATH  Google Scholar 

  4. Alfeo, A.L., Cimino, M.G., Vaglini, G.: Technological troubleshooting based on sentence embedding with deep transformers. J. Intell. Manuf. 32(6), 1699–1710 (2021)

    Article  Google Scholar 

  5. Alfeo, A.L., Cimino, M.G., Vaglini, G.: Degradation stage classification via interpretable feature learning. J. Manuf. Syst. 62, 972–983 (2022)

    Article  Google Scholar 

  6. Alfeo, A.L., Cimino, M.G.C.A., Gagliardi, G.: Concept-wise granular computing for explainable artificial intelligence. Granular Computing pp. 1–12 (2022)

    Google Scholar 

  7. Alfeo, A.L., Cimino, M.G.C., , Gagliardi, G.: Automatic feature extraction for bearings’ degradation assessment using minimally pre-processed time series and multi-modal feature learning. In: Proceedings of the 3rd International inproceedings on Innovative Intelligent Industrial Production and Logistics (IN4PL 2022) (2022)

    Google Scholar 

  8. Alfeo, A.L., Cimino, M.G.C.A., Egidi, S., Lepri, B., Pentland, A., Vaglini, G.: Stigmergy-based modeling to discover urban activity patterns from positioning data. In: Lee, D., Lin, Y.-R., Osgood, N., Thomson, R. (eds.) SBP-BRiMS 2017. LNCS, vol. 10354, pp. 292–301. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60240-0_35

    Chapter  Google Scholar 

  9. Alfeo, A.L., Zippo, A.G., Catrambone, V., Cimino, M.G., Toschi, N., Valenza, G.: From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks. Comput. Meth. Programs Biomed. 107550 (2023). https://doi.org/10.1016/j.cmpb.2023.107550

  10. Ali, S., et al.: Explainable artificial intelligence (XAI): what we know and what is left to attain trustworthy artificial intelligence. Inf. Fusion 101805 (2023)

    Google Scholar 

  11. Altmann, A., Toloşi, L., Sander, O., Lengauer, T.: Permutation importance: a corrected feature importance measure. Bioinformatics 26(10), 1340–1347 (2010)

    Article  PubMed  Google Scholar 

  12. Arras, L., Osman, A., Samek, W.: Clevr-XAI: a benchmark dataset for the ground truth evaluation of neural network explanations. Inf. Fusion 81, 14–40 (2022)

    Article  MATH  Google Scholar 

  13. Barr, B., Xu, K., Silva, C., Bertini, E., Reilly, R., Bruss, C.B., Wittenbach, J.D.: Towards ground truth explainability on tabular data. arXiv preprint arXiv:2007.10532 (2020)

  14. Bay, S.D., Kibler, D., Pazzani, M.J., Smyth, P.: The UCI KDD archive of large data sets for data mining research and experimentation. ACM SIGKDD Explorations Newsl 2(2), 81–85 (2000)

    Article  MATH  Google Scholar 

  15. Crupi, R., Castelnovo, A., Regoli, D., San Miguel Gonzalez, B.: Counterfactual explanations as interventions in latent space. In: Data Mining and Knowledge Discovery pp. 1–37 (2022)

    Google Scholar 

  16. Delaney, E., Greene, D., Keane, M.T.: Instance-based counterfactual explanations for time series classification. In: Sánchez-Ruiz, A.A., Floyd, M.W. (eds.) ICCBR 2021. LNCS (LNAI), vol. 12877, pp. 32–47. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-86957-1_3

    Chapter  MATH  Google Scholar 

  17. Galhotra, S., Pradhan, R., Salimi, B.: Feature attribution and recourse via probabilistic contrastive counterfactuals. In: Proceedings of the ICML Workshop on Algorithmic Recourse, pp. 1–6 (2021)

    Google Scholar 

  18. Guidotti, R.: Evaluating local explanation methods on ground truth. Artif. Intell. 291, 103428 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  19. Guidotti, R.: Counterfactual explanations and how to find them: literature review and benchmarking. In: Data Mining and Knowledge Discovery, pp. 1–55 (2022)

    Google Scholar 

  20. Horel, E., Giesecke, K.: Computationally efficient feature significance and importance for predictive models. In: Proceedings of the Third ACM International Conference on AI in Finance, pp. 300–307 (2022)

    Google Scholar 

  21. İç, Y.T., Yurdakul, M.: Development of a new trapezoidal fuzzy ahp-topsis hybrid approach for manufacturing firm performance measurement. Granular Comput. 6(4), 915–929 (2021)

    Article  MATH  Google Scholar 

  22. Jeyakumar, J.V., Noor, J., Cheng, Y.H., Garcia, L., Srivastava, M.: How can i explain this to you? an empirical study of deep neural network explanation methods. Adv. Neural. Inf. Process. Syst. 33, 4211–4222 (2020)

    MATH  Google Scholar 

  23. Kommiya Mothilal, R., Mahajan, D., Tan, C., Sharma, A.: Towards unifying feature attribution and counterfactual explanations: Different means to the same end. In: Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pp. 652–663 (2021)

    Google Scholar 

  24. Kumar, I.E., Venkatasubramanian, S., Scheidegger, C., Friedler, S.: Problems with shapley-value-based explanations as feature importance measures. In: International in Proceedings on Machine Learning, pp. 5491–5500. PMLR (2020)

    Google Scholar 

  25. Laugel, T., Renard, X., Lesot, M.J., Marsala, C., Detyniecki, M.: Defining locality for surrogates in post-hoc interpretablity. In: Workshop on Human Interpretability for Machine Learning (WHI)-International Conference on Machine Learning (ICML) (2018)

    Google Scholar 

  26. Liu, G., Yang, C., Liu, S., Xiao, C., Song, B.: Feature selection method based on mutual information and support vector machine. Int. J. Pattern Recognit Artif Intell. 35(06), 2150021 (2021)

    Article  MATH  Google Scholar 

  27. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30, pp. 4765–4774. Curran Associates, Inc. (2017). http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf

  28. Marcílio, W.E., Eler, D.M.: From explanations to feature selection: assessing shap values as feature selection mechanism. In: 2020 33rd SIBGRAPI conference on Graphics, Patterns and Images (SIBGRAPI), pp. 340–347. IEEE (2020)

    Google Scholar 

  29. Markus, A.F., Kors, J.A., Rijnbeek, P.R.: The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies. J. Biomed. Inform. 113, 103655 (2021)

    Article  PubMed  MATH  Google Scholar 

  30. Mosca, E., Szigeti, F., Tragianni, S., Gallagher, D., Groh, G.: Shap-based explanation methods: a review for NLP interpretability. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 4593–4603 (2022)

    Google Scholar 

  31. Mothilal, R.K., Sharma, A., Tan, C.: Explaining machine learning classifiers through diverse counterfactual explanations. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 607–617 (2020)

    Google Scholar 

  32. Pat, N., Wang, Y., Bartonicek, A., Candia, J., Stringaris, A.: Explainable machine learning approach to predict and explain the relationship between task-based FMRI and individual differences in cognition. bioRxiv pp. 2020–10 (2022)

    Google Scholar 

  33. Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: High-precision model-agnostic explanations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)

    Google Scholar 

  34. Setzu, M., Guidotti, R., Monreale, A., Turini, F., Pedreschi, D., Giannotti, F.: Glocalx-from local to global explanations of black box AI models. Artif. Intell. 294, 103457 (2021)

    Article  MathSciNet  Google Scholar 

  35. Sokol, K., Santos-Rodriguez, R., Flach, P.: Fat forensics: a python toolbox for algorithmic fairness, accountability and transparency. Software Impacts 14, 100406 (2022)

    Article  Google Scholar 

  36. Stepin, I., Alonso, J.M., Catala, A., Pereira-Fariña, M.: A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence. IEEE Access 9, 11974–12001 (2021)

    Article  MATH  Google Scholar 

  37. Urbanowicz, R.J., Meeker, M., La Cava, W., Olson, R.S., Moore, J.H.: Relief-based feature selection: introduction and review. J. Biomed. Inform. 85, 189–203 (2018)

    Article  PubMed  PubMed Central  MATH  Google Scholar 

  38. Vlassopoulos, G., van Erven, T., Brighton, H., Menkovski, V.: Explaining predictions by approximating the local decision boundary. arXiv preprint arXiv:2006.07985 (2020)

  39. Wiratunga, N., Wijekoon, A., Nkisi-Orji, I., Martin, K., Palihawadana, C., Corsar, D.: Actionable feature discovery in counterfactuals using feature relevance explainers. In: CEUR Workshop Proceedings (2021)

    Google Scholar 

  40. Yang, M., Kim, B.: Benchmarking attribution methods with relative feature importance. arXiv preprint arXiv:1907.09701 (2019)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Antonio Luca Alfeo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alfeo, A.L., Cimino, M.G.C.A., Gagliardi, G. (2025). Matching the Expert’s Knowledge via a Counterfactual-Based Feature Importance Measure. In: Meo, R., Silvestri, F. (eds) Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2023. Communications in Computer and Information Science, vol 2135. Springer, Cham. https://doi.org/10.1007/978-3-031-74633-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-74633-8_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-74632-1

  • Online ISBN: 978-3-031-74633-8

  • eBook Packages: Artificial Intelligence (R0)

Publish with us

Policies and ethics