Abstract
Neural networks are becoming increasingly better at tasks that involve classifying and recognizing images. At the same time techniques intended to explain the network output have been proposed. Here we examine three such techniques: Gradient-based Class Activation Mapping (Grad-CAM), Integrated Gradients (IG), and Integrated Grad-CAM, and introduce a new technique, that we call Riemann-Stieltjes Integrated Grad-CAM (RSI-Grad-CAM) that overcomes some of the shortcomings of those and similar techniques. Like Grad-CAM, our method can be applied to any layer of the network, and like Integrated Gradients it is not affected by the problem of vanishing gradients. For efficiency, gradient integration is performed numerically at the layer level using a Riemann-Stieltjes sum approximation. Compared to Grad-CAM, heatmaps produced by our algorithm are better focused in the areas of interest, and their numerical computation is more stable.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We will be using the terms heatmap, saliency map, and localization map interchangeably.
References
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I.J., Hardt, M., Kim, B.: Sanity checks for saliency maps. CoRR abs/1810.03292 (2018). https://arxiv.org/abs/1810.03292
Chattopadhyay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018). https://doi.org/10.1109/wacv.2018.00097
Chen, L., Chen, J., Hajimirsadeghi, H., Mori, G.: Adapting Grad-CAM for embedding networks (2020). https://arxiv.org/abs/2001.06538
Dhamdhere, K., Sundararajan, M., Yan, Q.: How important is a neuron? (2018). https://arxiv.org/abs/1805.12233
Kokhlikyan, N., et al.: Captum: a unified and generic model interpretability library for PyTorch (2020). https://arxiv.org/abs/2009.07896
Protter, M.H., Morrey, C.B.: The Riemann—Stieltjes integral and functions of bounded variation. In: A First Course in Real Analysis. Undergraduate Texts in Mathematics. Springer, NY (1991). https://doi.org/10.1007/978-1-4419-8744-0_12
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: explaining the predictions of any classifier. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations (2016). https://doi.org/10.18653/v1/N16-3020
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Sattarzadeh, S., Sudhakar, M., Plataniotis, K.N., Jang, J., Jeong, Y., Kim, H.: Integrated grad-CAM: sensitivity-aware visual explanation of deep convolutional networks via integrated gradient-based scoring (2021). https://arxiv.org/abs/2102.07805
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128(2), 336–359 (2019). https://doi.org/10.1007/s11263-019-01228-7
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) ICLR (2015). https://dblp.uni-trier.de/db/conf/iclr/iclr2015.html#SimonyanZ14a
Sorokin, A., Forsyth, D.A.: Utility data annotation with amazon mechanical turk. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–8 (2008)
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 3319–3328. PMLR (2017). https://proceedings.mlr.press/v70/sundararajan17a.html
Wang, H., et al.: Score-CAM: score-weighted visual explanations for convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 24–25 (2020)
Williamson, R., Trotter, H.: Multivariable Mathematics, 4th edn. Pearson Education Inc. (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lucas, M., Lerma, M., Furst, J., Raicu, D. (2022). RSI-Grad-CAM: Visual Explanations from Deep Networks via Riemann-Stieltjes Integrated Gradient-Based Localization. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13598. Springer, Cham. https://doi.org/10.1007/978-3-031-20713-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-20713-6_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20712-9
Online ISBN: 978-3-031-20713-6
eBook Packages: Computer ScienceComputer Science (R0)