Skip to main content
SpringerLink
Log in
Book cover

International Conference on Machine Learning, Optimization, and Data Science

LOD 2019: Machine Learning, Optimization, and Data Science pp 36–44Cite as

An Information Analysis Approach into Feature Understanding of Convolutional Deep Neural Networks

  • Conference paper
  • First Online:

  • 1784 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11943))

Abstract

This paper is centered on feature analysis and visualization of pre-trained deep neural networks based on responses of neurons to input images. In order to address this problem, first the information content of learned encodings of neurons is investigated based on the calculation of the salient activation map of each neuron. The salient activation map is considered to be the activation map that has the highest aggregative value over all its cells. Second, neurons are visualized based on their maximum activation response at each location. The results put forward the uncertainty reduction over the stage of deeper layers as well as a decrement pattern in Variation of Information.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    I also tried VGG19 model and obtained similar results.

References

  1. Bilal, A., Jourabloo, A., Ye, M., Liu, X., Ren, L.: Do convolutional neural networks learn class hierarchy? IEEE Trans. Visual Comput. Graph. 24(1), 152 (2018)

    Article  Google Scholar 

  2. Erhan, D., Bengio, Y., Courville, A., Vincent, P.: Visualizing higher-layer features of a deep network. Univ. Montreal 1341(3), 1 (2009)

    Google Scholar 

  3. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 38(1), 142–158 (2016)

    Article  Google Scholar 

  4. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  5. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

    Google Scholar 

  6. Long, J.L., Zhang, N., Darrell, T.: Do convnets learn correspondence? In: Advances in Neural Information Processing Systems, pp. 1601–1609 (2014)

    Google Scholar 

  7. Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5188–5196 (2015)

    Google Scholar 

  8. Montavon, G., Samek, W., Müller, K.R.: Methods for interpreting and understanding deep neural networks. Digit. Signal Proc. 73, 1–15 (2018)

    Article  MathSciNet  Google Scholar 

  9. Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., Clune, J.: Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In: Advances in Neural Information Processing Systems, pp. 3387–3395 (2016)

    Google Scholar 

  10. Nguyen, A., Yosinski, J., Clune, J.: Understanding neural networks via feature visualization: a survey. arXiv preprint arXiv:1904.08939 (2019)

  11. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems, pp. 3856–3866 (2017)

    Google Scholar 

  12. Sadeghi, Z.: Deep learning and developmental learning: emergence of fine-to-coarse conceptual categories at layers of deep belief network. Perception 45(9), 1036–1045 (2016)

    Article  Google Scholar 

  13. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)

  14. Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579 (2015)

  15. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53

    Chapter  Google Scholar 

  16. Zeiler, M.D., Krishnan, D., Taylor, G.W., Fergus, R.: Deconvolutional networks (2010)

    Google Scholar 

  17. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Object detectors emerge in deep scene CNNs. arXiv preprint arXiv:1412.6856 (2014)

  18. Zintgraf, L.M., Cohen, T.S., Welling, M.: A new method to visualize deep neural networks. arXiv preprint arXiv:1603.02518 (2016)

Download references

Acknowledgment

This research has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 665919.

Author information

Authors and Affiliations

  1. Universitat Autonoma de Barcelona, Barcelona, Spain

    Zahra Sadeghi

Authors
  1. Zahra Sadeghi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zahra Sadeghi .

Editor information

Editors and Affiliations

  1. University of Cambridge, Cambridge, UK

    Giuseppe Nicosia

  2. University of Florida, Gainesville, FL, USA

    Panos Pardalos

  3. Harvard University, Cambridge, MA, USA

    Renato Umeton

  4. Università di Catania, Catania, Catania, Italy

    Giovanni Giuffrida

  5. Almawave, Rome, Roma, Italy

    Vincenzo Sciacca

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sadeghi, Z. (2019). An Information Analysis Approach into Feature Understanding of Convolutional Deep Neural Networks. In: Nicosia, G., Pardalos, P., Umeton, R., Giuffrida, G., Sciacca, V. (eds) Machine Learning, Optimization, and Data Science. LOD 2019. Lecture Notes in Computer Science(), vol 11943. Springer, Cham. https://doi.org/10.1007/978-3-030-37599-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37599-7_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37598-0

  • Online ISBN: 978-3-030-37599-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation