Skip to main content
SpringerLink
Log in

Causal Explanation of Convolutional Neural Networks

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases. Research Track (ECML PKDD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12976))

Abstract

In this paper we introduce an explanation technique for Convolutional Neural Networks (CNNs) based on the theory of causality by Halpern and Pearl [12]. The causal explanation technique (CexCNN) is based on measuring the filter importance to a CNN decision, which is measured through counterfactual reasoning. In addition, we employ extended definitions of causality, which are responsibility and blame to weight the importance of such filters and project their contribution on input images. Since CNNs form a hierarchical structure, and since causal models can be hierarchically abstracted, we employ this similarity to perform the most important contribution of this paper, which is localizing the important features in the input image that contributed the most to a CNN’s decision. In addition to its ability in localization, we will show that CexCNN can be useful as well for model compression through pruning the less important filters. We tested CexCNN on several CNNs architectures and datasets. (The code is available on https://github.com/HichemDebbi/CexCNN)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Investigate. https://github.com/albermax/innvestigate

  2. keras-surgeon. https://github.com/BenWhetton/keras-surgeon

  3. Keras visualization toolkit. https://github.com/raghakot/keras-vis

  4. Avanti, S., Peyton, G., Anshul, K.: Learning important features through propagating activation differences, pp. 3145–3153. ICML’17 (2017)

    Google Scholar 

  5. Beckers, S., Halpern, J.Y.: Abstracting causal models. In: AAAI (2017)

    Google Scholar 

  6. Chockler, H., Halpern, J.Y.: Responsibility and blame: a structural-model approach. J. Artif. Int. Res. 22(1), 93–115 (2004)

    MathSciNet  MATH  Google Scholar 

  7. Choe, J., Oh, S.J., Lee, S., Chun, S., Akata, Z., Shim, H.: Evaluating weakly supervised object localization methods right. In: CVPR, pp. 3130–3139 (2020)

    Google Scholar 

  8. Gordon, D., Kembhavi, A., Rastegari, M., Redmon, J., Fox, D., Farhadi, A.: Iqa: visual question answering in interactive environments. In: In arXiv:1712.03316 (2017)

  9. Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)

    Google Scholar 

  10. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR, pp. 580–587 (2014)

    Google Scholar 

  11. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: ICLR (2015)

    Google Scholar 

  12. Halpern, J., Pearl, J.: Causes and explanations: a structural-model approach part i: Causes. In: Proceedings of the 17th UAI, pp. 194–202 (2001)

    Google Scholar 

  13. Halpern, J.Y., Pearl, J.: Causes and explanations: a structural-model approach. part ii: Explanations. Br. J. Philos. Sci. 56(4), 889–911 (2008)

    Google Scholar 

  14. Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. In: NIPS (2015)

    Google Scholar 

  15. Harradon, M., Druce, J., Ruttenberg, B.E.: Causal learning and explanation of deep neural networks via autoencoded activations. In: CoRR abs/1802.00541 (2018)

    Google Scholar 

  16. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  17. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  18. LeCun, Y., Cortes, C., Burges, C.: Mnist handwritten digit database. ATT Labs [Online]. http://yann.lecun.com/exdb/mnist 2 (2010)

  19. Lewis, D.: Causation. J. Philos. 70, 556–567 (1972)

    Article  Google Scholar 

  20. Li, H., Kadav, A., Durdanovic, I., Samety, H.: Pruning filters for efficient convnets. In: ICLR 2017, pp. 1–13 (2017)

    Google Scholar 

  21. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR, pp. 3431–3440 (2015)

    Google Scholar 

  22. Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: NIPS, pp. 4768–4777 (2017)

    Google Scholar 

  23. Molnar, C.: Interpretable Machine Learning A Guide for Making Black Box Models Explainable (2018). https://christophm.github.io/interpretable-ml-book/

  24. Narendra, T., Sankaran, A., Vijaykeerthy, D., Mani, S.: Explaining deep learning models using causal inference. In: arXiv:1811.04376 (2018)

  25. Ribeiro, M.T., Singh, S., Guestrin, C.: “why should i trust you?": Explaining the predictions of any classifier, pp. 1135–1144. KDD ’16 (2016)

    Google Scholar 

  26. Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)

    Article  MathSciNet  Google Scholar 

  27. Schwab, P., Karlen, W.: Cxplain: causal explanations for model interpretation under uncertainty. NeurIPS, pp. 10220–10230 (2019)

    Google Scholar 

  28. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: ICCV, pp. 618–626 (2017)

    Google Scholar 

  29. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: In arXiv:1312.6034 (2013)

  30. Singh, K.K., Lee, Y.J.: Forcing a network to be meticulous for weakly-supervised object and action localization. In: CVPR (2017)

    Google Scholar 

  31. Smilkov, D., Thorat, N., Kim, B., Viegas, F.B., Wattenberg, M.: Smoothgrad: removing noise by adding noise. In: CoRR, vol. abs/1706.03825 (2017)

    Google Scholar 

  32. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: ICML, pp. 3319–3328 (2017)

    Google Scholar 

  33. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  34. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: CVPR, pp. 2921–2929 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of M’sila, M’sila, Algeria

    Hichem Debbi

Authors
  1. Hichem Debbi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hichem Debbi .

Editor information

Editors and Affiliations

  1. ELLIS - The European Laboratory for Learning and Intelligent Systems, Alicante, Spain

    Nuria Oliver

  2. ETHZ and EPFL, Zürich, Switzerland

    Fernando Pérez-Cruz

  3. Johannes Gutenberg University of Mainz, Mainz, Germany

    Stefan Kramer

  4. École Polytechnique, Palaiseau, France

    Jesse Read

  5. Basque Center for Applied Mathematics, Bilbao, Spain

    Jose A. Lozano

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Debbi, H. (2021). Causal Explanation of Convolutional Neural Networks. In: Oliver, N., Pérez-Cruz, F., Kramer, S., Read, J., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2021. Lecture Notes in Computer Science(), vol 12976. Springer, Cham. https://doi.org/10.1007/978-3-030-86520-7_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86520-7_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86519-1

  • Online ISBN: 978-3-030-86520-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation