Skip to main content
SpringerLink
Log in

Log-Nets: Logarithmic Feature-Product Layers Yield More Compact Networks

  • Conference paper
  • First Online:
Book cover Artificial Neural Networks and Machine Learning – ICANN 2020 (ICANN 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12397))

Included in the following conference series:

Abstract

We introduce Logarithm-Networks (Log-Nets), a novel bio-inspired type of network architecture based on logarithms of feature maps followed by convolutions. Log-Nets are capable of surpassing the performance of traditional convolutional neural networks (CNNs) while using fewer parameters. Performance is evaluated on the Cifar-10 and ImageNet benchmarks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Barth, E., Watson, A.B.: A geometric framework for nonlinear visual coding. Opt. Express 7(4), 155–165 (2000)

    Article  Google Scholar 

  2. Bergstra, J., Desjardins, G., Lamblin, P., Bengio, Y.: Quadratic polynomials learn better image features. Technical report, 1337 (2009)

    Google Scholar 

  3. Berkes, P., Wiskott, L.: On the analysis and interpretation of inhomogeneous quadratic forms as receptive fields. Neural Comput. 18, 1868–1895 (2006)

    Article  MathSciNet  Google Scholar 

  4. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR 2009 (2009)

    Google Scholar 

  5. Frankle, J., Carbin, M.: The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018)

  6. HasanPour, S.H., Rouhani, M., Fayyaz, M., Sabokrou, M.: Lets keep it simple, using simple architectures to outperform deeper and more complex architectures. arXiv preprint arXiv:1608.06037 (2016)

  7. Hinton, G.F.: A parallel computation that assigns canonical object-based frames of reference. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, vol. 2, pp. 683–685 (1981)

    Google Scholar 

  8. Howard, J., et al.: fast.ai (2018). https://github.com/fastai/imagenet-fast/blob/master/imagenet_nv/fastai_imagenet.py

  9. Huang, Y., et al.: GPipe: efficient training of giant neural networks using pipeline parallelism. In: Advances in Neural Information Processing Systems, pp. 103–112 (2019)

    Google Scholar 

  10. Hubel, D.H., Wiesel, T.N.: Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol. 28(2), 229–289 (1965)

    Article  Google Scholar 

  11. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)0.5 mb model size. arXiv preprint arXiv:1602.07360 (2016)

  12. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)

    Google Scholar 

  13. Krizhevsky, A., Nair, V., Hinton, G.: CIFAR-10 (Canadian Institute for Advanced Research). http://www.cs.toronto.edu/~kriz/cifar.html

  14. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  15. kuangliu: pytorch-cifar (2020). https://github.com/kuangliu/pytorch-cifar/blob/master/models/efficientnet.py

  16. Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016)

  17. Mota, C., Barth, E.: On the uniqueness of curvature features. In: Baratoff, G., Neumann, H. (eds.) Dynamische Perzeption. Proceedings in Artificial Intelligence, Köln, vol. 9, pp. 175–178 (2000)

    Google Scholar 

  18. Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019)

    Google Scholar 

  19. Poon, H., Domingos, P.: Sum-product networks: a new deep architecture. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 689–690. IEEE (2011)

    Google Scholar 

  20. Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems, pp. 3856–3866 (2017)

    Google Scholar 

  21. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  22. Shridhar, K., Laumann, F., Liwicki, M.: A comprehensive guide to Bayesian convolutional neural network with variational inference. arXiv preprint arXiv:1901.02731 (2019)

  23. Springenberg, J., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net. In: ICLR (workshop track) (2015)

    Google Scholar 

  24. Stojnic, R., Taylor, R.: ImageNet leaderboard (2020). https://paperswithcode.com/sota/image-classification-on-imagenet

  25. Tan, M., et al.: MnasNet: platform-aware neural architecture search for mobile. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2820–2828 (2019)

    Google Scholar 

  26. Tan, M., Le, Q.: EfficientNet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019)

    Google Scholar 

  27. Tan, M., Le, Q.V.: MixConv: Mixed depthwise convolutional kernels. CoRR, abs/1907.09595 (2019)

    Google Scholar 

  28. Veniat, T., Denoyer, L.: Learning time/memory-efficient deep architectures with budgeted super networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3492–3500 (2018)

    Google Scholar 

  29. Volterra, V.: Theory of Functionals and of Integral and Integro-differential Equations. Dover Publications, Mineola, New York (1959)

    MATH  Google Scholar 

  30. Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., Hu, Q.: ECA-Net: Efficient channel attention for deep convolutional neural networks. arXiv preprint arXiv:1910.03151 (2019)

  31. Xie, L., Yuille, A.: Genetic CNN. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1379–1388 (2017)

    Google Scholar 

  32. Zagoruyko, S., Komodakis, N.: Wide residual networks. arXiv preprint arXiv:1605.07146 (2016)

  33. Zetzsche, C., Barth, E.: Fundamental limits of linear filters in the visual processing of two-dimensional signals. Vis. Res. 30, 1111–1117 (1990)

    Article  Google Scholar 

  34. Zetzsche, C., Barth, E., Wegmann, B.: The importance of intrinsically two-dimensional image features in biological vision and picture coding. In: Watson, A.B. (ed.) Digital Images and Human Vision, pp. 109–38. MIT Press (October 1993)

    Google Scholar 

  35. Zhang, X., Zhou, X., Lin, M., Sun, J.: ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)

    Google Scholar 

  36. Zoumpourlis, G., Doumanoglou, A., Vretos, N., Daras, P.: Non-linear convolution filters for CVV-based learning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4761–4769 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Neuro- and Bioinformatics, University of Lübeck, Lübeck, Germany

    Philipp Grüning, Thomas Martinetz & Erhardt Barth

Authors
  1. Philipp Grüning
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Martinetz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erhardt Barth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Grüning .

Editor information

Editors and Affiliations

  1. Department of Applied Informatics, Comenius University in Bratislava, Bratislava, Slovakia

    Igor Farkaš

  2. Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kgs. Lyngby, Denmark

    Paolo Masulli

  3. Department of Informatics, University of Hamburg, Hamburg, Germany

    Stefan Wermter

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Grüning, P., Martinetz, T., Barth, E. (2020). Log-Nets: Logarithmic Feature-Product Layers Yield More Compact Networks. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12397. Springer, Cham. https://doi.org/10.1007/978-3-030-61616-8_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-61616-8_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-61615-1

  • Online ISBN: 978-3-030-61616-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation