Abstract
When determining a lung nodule malignancy one must consider the spiculation represented by spike-like structures in the nodule’s boundary. In this paper, we develop a deep learning model based on a VGG16 architecture to locate the presence of spiculation in lung nodules from Computed Tomography images. In order to increase the expert’s confidence in the model output, we apply our novel Riemann-Stieltjes Integrated Gradient-weighted Class Activation Mapping attribution method to visualize areas of the image (spicules). Therefore, the attribution method is applied to the layer of the model that is responsible for the detection of the spiculation features. We show that the first layers of the network are specialized in detecting low-level features such as edges, the last convolutional layer detects the general area occupied by the nodule, and finally, we identify that spiculation structures are detected at an intermediate layer. We use three different metrics to support our findings.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Andrejeva, L., Geisel, J.L., Harigopa, M.: Spiculated masses, breast imaging. Oxford Medicine Online (2018). https://doi.org/10.1093/med/9780190270261.003.0025
Bengio, Y.: Deep learning of representations for unsupervised and transfer learning. In: Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning Workshop, vol. 27, pp. 17–37 (2011)
Chattopadhyay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018). https://doi.org/10.1109/wacv.2018.00097
Devore, J.L., Berk, K.N.: Discrete random variables and probability distributions. In: Devore, J.L., Berk, K.N. (eds.) Modern Mathematical Statistics with Applications. STS, pp. 96–157. Springer, New York (2012). https://doi.org/10.1007/978-1-4614-0391-3_3
Hancock, M.C., Magnan, J.F.: Lung nodule malignancy classification using only radiologist-quantified image features as inputs to statistical learning algorithms: probing the lung image database consortium dataset with two statistical learning methods. J. Med. Imaging 3(4), 044504 (2016)
Hochreite, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl.-Based Syst. 06(02), 107–116 (1998)
Hofer, M.: CT Teaching Manual, A Systematic Approach to CT Reading. Thieme Publishing Group (2007)
Huanga, S., Yang, J., Fong, S., Zhao, Q.: Artificial intelligence in cancer diagnosis and prognosis: opportunities and challenges. Cancer Lett. 471(28), 61–71 (2020)
Armato III, S.G., et al.: Lung image database consortium: developing a resource for the medical imaging research community. Radiology 232(3), 739–748 (2004). https://doi.org/10.1148/radiol.2323032035
Kantorovich, L.V.: Mathematical methods of organizing and planning production. Manage. Sci. 6(4), 366–422 (1939)
Lao, Z., Zheng, X.: Multiscale quantification of tissue spiculation and distortion for detection of architectural distortion and spiculated mass in mammography. In: M.D., R.M.S., van Ginneken, B. (eds.) Medical Imaging 2011: Computer-Aided Diagnosis, vol. 7963, pp. 468–475. International Society for Optics and Photonics, SPIE (2011). https://doi.org/10.1117/12.877330
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)
Lucas, M., Lerma, M., Furst, J., Raicu, D.: Visual explanations from deep networks via Riemann-Stieltjes integrated gradient-based localization (2022). https://arxiv.org/abs/2205.10900
Mann, H., Whitney, D.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18(1), 50–60 (1947)
Mendez, D.M.M., Bermúdez, A., Tyrrell, P.N.: Visualization of layers within a convolutional neural network using gradient activation maps. J. Undergraduate Life Sci. 14(1), 6 (2020)
Protter, M.H., Morrey, C.B.: The Riemann-Stieltjes integral and functions of bounded variation. In: Protter, M.H., Morrey, C.B. (eds.) A First Course in Real Analysis. Undergraduate Texts in Mathematics. Springer, New York (1991). https://doi.org/10.1007/978-1-4419-8744-0_12
Nadeem, W.C.S., Alam, S.R., Deasy, J.O., Tannenbaum, A., Lu, W.: Reproducible and interpretable spiculation quantification for lung cancer screening. Comput. Methods Programs Biomed. (2020). https://doi.org/10.1016/j.cmpb.2020.105839
Paci, E., et al.: Ma01.09 mortality, survival and incidence rates in the italung randomised lung cancer screening trial (Italy). J. Thorac. Oncol. 12(1), S346–S347 (2017)
Qiu, B., Furst, J., Rasin, A., Tchoua, R., Raicu, D.: Learning latent spiculated features for lung nodule characterization. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1254–1257 (2020). https://doi.org/10.1109/EMBC44109.2020.9175720
Qiu, S., Sun, J., Zhou, T., Gao, G., He, Z., Liang, T.: Spiculation sign recognition in a pulmonary nodule based on spiking neural p systems. BioMed Res. Int. 2020 (2020). https://doi.org/10.1155/2020/6619076
Ramdas, A., Garcia, N., Cuturi, M.: On Wasserstein two sample testing and related families of nonparametric tests. Entropy 19(2), 47 (2017). https://doi.org/10.3390/e19020047
Rizzo, M.L., Székely, G.J.: Energy distance. Wiley Interdiscip. Rev. Comput. Stat. 8(1), 27–38 (2015)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y
Sattarzadeh, S., Sudhakar, M., Plataniotis, K.N., Jang, J., Jeong, Y., Kim, H.: Integrated Grad-CAM: sensitivity-aware visual explanation of deep convolutional networks via integrated gradient-based scoring (2021). https://arxiv.org/abs/2102.07805
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vision 128(2), 336–359 (2019). https://doi.org/10.1007/s11263-019-01228-7
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015). https://arxiv.org/abs/1409.1556
Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 3319–3328. PMLR (2017). https://proceedings.mlr.press/v70/sundararajan17a.html
Szekely, G.J.: E-statistics: the energy of statistical samples. Technical report, Bowling Green State University, Department of Mathematics and Statistics (2002)
Waserstein, L.N.: Markov processes over denumerable products of spaces, describing large systems of automata. Problemy Peredac̆i Informacii 5(3), 6–72 (1969)
Winkels, M., Cohena, T.S.: Pulmonary nodule detection in CT scans with equivariant CNNs. Med. Image Anal. 55, 15–26 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lucas, M., Lerma, M., Furst, J., Raicu, D. (2023). Explainable Model for Localization of Spiculation in Lung Nodules. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13807. Springer, Cham. https://doi.org/10.1007/978-3-031-25082-8_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-25082-8_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25081-1
Online ISBN: 978-3-031-25082-8
eBook Packages: Computer ScienceComputer Science (R0)