Skip to main content

Advertisement

Springer Nature Link
Log in

A Novel Multi-Type Image Coding Method Acting on Supervised Hierarchical Deep Spiking Convolutional Neural Networks for Image Classification

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Spiking neural networks (SNNs) have gained significant momentum in recent times as they transmit information via discrete spikes, similar to neuromorphic low-power systems. However, existing spike coding methods are often limited to a single scale of time or rate, and typically suffer from drawbacks such as reduced accuracy or long classification latency. In this paper, we propose a pixel-based multi-type image coding (PMIC) method inspired by the functional organization of primate visual systems to address the issues at hand. The encoded information comprises both spatial and temporal details, represented by spiking firing time and intensity, respectively. Subsequently, we combine the spiking firing time and intensity as inputs of a hierarchical spiking convolutional neural network (SCNN) including several convolutional and pooling layers. During the training phase, we use error backpropagation to optimize parameters. Comparison of experimental results with some state-of-the-art approaches on MNIST dataset, Fashion-MNIST dataset as well as ETH-80 dataset of image classification demonstrates that SCNN using PMIC can achieve the best test accuracy, which is 99.13%, 90.31%, and 94.29%, respectively. The proposed PMIC utilizes multiple filters and coding strategies to extract multi-type information and is more beneficial to the performance of SNNs compared to methods that extract single-scale or single-type information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Huang K-Q, Ren W-Q, Tan T-N. A review on image object classification and detection. Chinese Journal of Computers. 2014;37(6):1225–40.

    MATH  Google Scholar 

  2. Bhosale YH, Patnaik KS. PulDi-COVID: chronic obstructive pulmonary (lung) diseases with COVID-19 classification using ensemble deep convolutional neural network from chest x-ray images to minimize severity and mortality rates. Biomedical Signal Processing and Control. 2023;81.

  3. Touvron H, Bojanowski P, Caron M, Cord M, El-Nouby A, Grave E, Izacard G, Joulin A, Synnaeve G, Verbeek J, et al. ResMLP: feedforward networks for image classification with data-efficient training. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022;45(4):5314–21.

    Google Scholar 

  4. Bhosale YH, Patnaik KS. Graph and capsule convolutional neural network based classification of lung cancer, pneumonia, COVID-19 using lung CT and ultrasound radiography imaging. In: 2022 8th International Conference on Signal Processing and Communication (ICSC), IEEE; 2022. pp. 381–387.

  5. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556.

  6. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015. pp. 1–9.

  7. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. pp. 770–778.

  8. Tan M, Le Q. Efficientnet: rethinking model scaling for convolutional neural networks, in: International conference on machine learning, PMLR; 2019. pp. 6105–6114.

  9. Lecun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436.

    MATH  Google Scholar 

  10. Li S, Mao Y, Zhang F, Wang D, Zhong G. DLW-NAS: differentiable light-weight neural architecture search, Cognitive Computation. 2022. 1–11.

  11. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge: MIT Press; 2016.

    MATH  Google Scholar 

  12. Lecun Y, Bottou L. Gradient-based learning applied to document recognition. Proceedings of the IEEE. 1998;86(11):2278–324.

    MATH  Google Scholar 

  13. Krizhevsky A, Hinton G., et al. Learning multiple layers of features from tiny images.

  14. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Communications of the ACM. 2017;60(6):84–90.

    MATH  Google Scholar 

  15. Bhosale YH, Patnaik KS. Application of deep learning techniques in diagnosis of COVID-19 (coronavirus): a systematic review, Neural Processing Letters. 2022; 1–53.

  16. Bhosale YH, Patnaik KS, Ecg-ccnet: Cardiovascular (cardiac) and covid-19 disease classification using deep convolutional neural network learning pipeline approaches from electrocardiography (ecg)-a study, in,. IEEE Silchar Subsection Conference (SILCON). IEEE. 2022;2022:1–6.

  17. Yao M, Gao H, Zhao G, Wang D, Lin Y, Yang Z, Li G. Temporal-wise attention spiking neural networks for event streams classification, in: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021, pp. 10221–10230.

  18. Stuijt J, Sifalakis M, Yousefzadeh A, Corradi F. \(\mu \)brain: an event-driven and fully synthesizable architecture for spiking neural networks. Frontiers in Neuroscience. 2021;15:538.

    Google Scholar 

  19. Voutsas K, Adamy J. A biologically inspired spiking neural network for sound source lateralization. IEEE Trans Neural Netw. 2007;18(6):1785–99.

    MATH  Google Scholar 

  20. Perez-Carrasco JA, Zhao B, Serrano C, Acha B. Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing-application to feedforward convnets. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2013;35(11):2706–19.

    Google Scholar 

  21. Kasabov NK, Doborjeh MG, Doborjeh ZG. Mapping, learning, visualization, classification, and understanding of fMRI data in the NeuCube evolving spatiotemporal data machine of spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems. 2017;28(4):887–99.

    MATH  Google Scholar 

  22. Jin Z. Dezhe, Fast convergence of spike sequences to periodic patterns in recurrent networks. Physical Review Letters. 2002;89(20): 208102.

    Google Scholar 

  23. Nádasdy Z. Spike sequences and their consequences. Journal of Physiology-Paris. 2000;94(5–6):505–24.

    MATH  Google Scholar 

  24. Auge D, Hille J, Mueller E, Knoll A. A survey of encoding techniques for signal processing in spiking neural networks. Neural Processing Letters. 2021;53(6):4693–710.

    MATH  Google Scholar 

  25. Stöckl C, Maass W. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence. 2021;3(3):230–8.

    MATH  Google Scholar 

  26. Guo W, Fouda ME, Eltawil AM, Salama KN. Neural coding in spiking neural networks: a comparative study for robust neuromorphic systems. Frontiers in Neuroscience. 2021;15.

  27. Rueckauer B, Liu S-C. Temporal pattern coding in deep spiking neural networks. In: 2021 International Joint Conference on Neural Networks (IJCNN), IEEE. 2021, pp. 1–8.

  28. Eurich CW, Wilke SD. Multidimensional encoding strategy of spiking neurons. Neural Computation. 2014;12(7):1519–29.

    MATH  Google Scholar 

  29. Cubero RJ, Marsili M, Roudi Y. Multiscale relevance and informative encoding in neuronal spike trains. Journal of computational neuroscience. 2020;48:85–102.

    MATH  Google Scholar 

  30. Zhou Z, Wang C, Feng Y, Chen D. Jointe: jointly utilizing 1d and 2d convolution for knowledge graph embedding. Knowledge-Based Systems. 2022;240.

  31. Qin W, Wen H, Li F. Fabric defect detection algorithm based on residual energy distribution and Gabor feature fusion. The Visual Computer. 2022; 1–15.

  32. Masood H, Zafar A, Ali MU, Hussain T, Khan MA, Tariq U, Damaševičius R. Tracking of a fixed-shape moving object based on the gradient descent method. Sensors. 2022;22(3):1098.

    MATH  Google Scholar 

  33. Hubel DH, Wiesel TN. Receptive fields of single neurones in the cat’s striate cortex. The Journal of physiology. 1959;148(3):574.

    MATH  Google Scholar 

  34. Adrian ED. The impulses produced by sensory nerve endings: part I. The Journal of physiology. 1926;61(1):49.

    MATH  Google Scholar 

  35. Kim Y, Park H, Moitra A, Bhattacharjee A, Venkatesha Y, Panda P. Rate coding or direct coding: which one is better for accurate, robust, and energy-efficient spiking neural networks?, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE. 2022, pp. 71–75.

  36. Jin Y, Zhang W, Li P. Hybrid macro/micro level backpropagation for training deep spiking neural networks. Advances in Neural Information Processing Systems 31.

  37. Engel AK, König P, Kreiter AK, Schillen TB, Singer W. Temporal coding in the visual cortex: new vistas on integration in the nervous system-sciencedirect. Trends in Neurosciences. 1992;15(6):218–26.

    Google Scholar 

  38. Bohte SM. The evidence for neural information processing with precise spike-times: a survey. Natural Computing. 2004;3(2):195–206.

    MathSciNet  MATH  Google Scholar 

  39. Knoblauch A, Hauser F. STDP, Hebbian cell assemblies, and temporal coding by spike synchronization. BMC Neuroscience. 2011;12(1):P142.

    MATH  Google Scholar 

  40. Heller J, Hertz JA, Kjr TW, Richmond BJ. Information flow and temporal coding in primate pattern vision. Journal of Computational Neuroscience. 1995;2(3):175–93.

    MATH  Google Scholar 

  41. Greenberg S, Marsh JT, Brown WS, Smith JC. Neural temporal coding of low pitch. I. Human frequency-following responses to complex tones. Hearing research. 1987;25(2):91.

    Google Scholar 

  42. W. Maas, Noisy spiking neurons with temporal coding have more computational power than sigmoidal neurons, Advances in Neural Information Processing Systems. 1997; 9 211–217.

  43. Rueckauer B, Liu, SC. Conversion of analog to spiking neural networks using sparse temporal coding, in: IEEE International Symposium on Circuits and Systems. 2018; pp. 1–5.

  44. Park S, Kim S, Na B, Yoon S, T2fsnn: deep spiking neural networks with time-to-first-spike coding. In,. 57th ACM/IEEE design automation conference (DAC). IEEE. 2020;2020:1–6.

  45. Wysoski SG, Benuskova L, Kasabov NK. Brain-like system for audiovisual person authentication based on time-to-first spike coding. In: Computational Modeling and Simulation of Intellect: Current State and Future Perspectives, IGI Global. 2011, pp. 384–412.

  46. Nadasdy Z, Information encoding and reconstruction by phase coding of spikes, Analysis and Modeling of Coordinated Multi-neuronal Activity. 2015; 269–298.

  47. Orhan AE, Ma WJ. Neural population coding of multiple stimuli. Journal of Neuroscience. 2015;35(9):3825–41.

    MATH  Google Scholar 

  48. Mayforth GR. An introduction to neural information processing. Computing reviews. 2016;57(8):481–2.

    MATH  Google Scholar 

  49. Lewen GD, Bialek W, Steveninck R. Neural coding of naturalistic stimuli. Network Computation in Neural Systems. 2001;12(3):317–29.

    MATH  Google Scholar 

  50. Liu YH, Wang XJ. Spike-frequency adaptation of a generalized leaky integrate-and-fire model neuron. Journal of Computational Neuroscience. 2001;10(1):25–45.

    MATH  Google Scholar 

  51. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR. Improving neural networks by preventing co-adaptation of feature detectors. Computer Science. 2012;3(4):212–23.

    MATH  Google Scholar 

  52. Serre T, Wolf L, Bileschi S, Riesenhuber M, Poggio T. Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2007;29:411–26.

    MATH  Google Scholar 

  53. Wu Y, Deng L, Li G, Zhu J, Shi L. Direct training for spiking neural networks: faster, larger, better. Proceedings of the AAAI Conference on Artificial Intelligence. 2019;33:1311–8.

    MATH  Google Scholar 

  54. Panda P, Roy K. Unsupervised regenerative learning of hierarchical features in spiking deep networks for object recognition. In: 2016 International Joint Conference on Neural Networks (IJCNN), 2016.

  55. Eshraghian JK, Lu WD. The fine line between dead neurons and sparsity in binarized spiking neural networks, arXiv preprint arXiv:2201.11915.

  56. Tang H, Cho D, Lew D, Kim T, Park J. Rank order coding based spiking convolutional neural network architecture with energy-efficient membrane voltage updates. Neurocomputing. 2020;407:300–12.

    MATH  Google Scholar 

  57. Wang T, Shi C, Zhou X, Lin Y, He J, Gan P, Li P, Wang Y, Liu L, Wu N, et al. CompSNN: a lightweight spiking neural network based on spatiotemporally compressive spike features. Neurocomputing. 2021;425:96–106.

    Google Scholar 

  58. Zhao B, Ding R, Chen S, Linares-Barranco B, Tang H. Feedforward categorization on AER motion events using cortex-like features in a spiking neural network. IEEE Transactions on Neural Networks and Learning Systems. 2017;26(9):1963–78.

    MathSciNet  MATH  Google Scholar 

  59. Querlioz D, Bichler O, Dollfus P, Gamrat C. Immunity to device variations in a spiking neural network with memristive nanodevices. Nanotechnology IEEE Transactions on. 2013;12(3):288–95.

  60. Peter O, Daniel N, Liu SC, Tobi D, Michael P. Real-time classification and sensor fusion with a spiking deep belief network. Frontiers in Neuroscience. 2013;7(7):178.

    Google Scholar 

  61. Peter U. Diehl, Matthew, Cook, Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in Computational Neuroscience. 2015;9:99.

    MATH  Google Scholar 

  62. Shi C, Wang T, He J, Zhang J, Liu L, Wu N. Deeptempo: a hardware-friendly direct feedback alignment multi-layer tempotron learning rule for deep spiking neural networks. IEEE Transactions on Circuits and Systems II: Express Briefs. 2021;68(5):1581–5.

    Google Scholar 

  63. Kheradpisheh SR, Mirsadeghi M, Masquelier T. Bs4nn: binarized spiking neural networks with temporal coding and learning. Neural Processing Letters. 2022;54(2):1255–73.

    MATH  Google Scholar 

  64. Tavanaei A, Maida A. BP-STDP: approximating backpropagation using spike timing dependent plasticity. Neurocomputing. 2019; 330(FEB.22), 39–47.

  65. Mozafari M, Ganjtabesh M, Nowzari-Dalini A, Thorpe SJ, Masquelier T. Bio-inspired digit recognition using spike-timing-dependent plasticity (STDP) and reward-modulated STDP in deep convolutional networks. https://arxiv.org/abs/180400227.

  66. Kheradpisheh SR, Masquelier T. Temporal backpropagation for spiking neural networks with one spike per neuron. International Journal of Neural Systems. 2020;30(06):2050027.

    MATH  Google Scholar 

  67. Comsa IM, Potempa K, Versari L, Fischbacher T. A. Gesmundo, J. Alakuijala, Temporal coding in spiking neural networks with alpha synaptic function, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 8529–8533.

  68. Zhao D, Li Y, Zeng Y, Wang J, Zhang Q. Spiking capsnet: a spiking neural network with a biologically plausible routing rule between capsules. Information Sciences. 2022;610:1–13.

  69. Kheradpisheh SR, Ganjtabesh M, Thorpe SJ, Masquelier T. STDP-based spiking deep convolutional neural networks for object recognition. Neural Networks. 2018;99:56–67.

  70. Zhang M, Wang J, Wu J, Belatreche A, Amornpaisannon B, Zhang Z, Miriyala VPK, Qu H, Chua Y, Carlson TE, et al. Rectified linear postsynaptic potential function for backpropagation in deep spiking neural networks. IEEE transactions on neural networks and learning systems. 2021;33(5):1947–58.

    Google Scholar 

  71. Zhao D, Zeng Y, Zhang T, Shi M, Zhao F. GLSNN: a multi-layer spiking neural network based on global feedback alignment and local STDP plasticity. Frontiers in Computational Neuroscience. 2020;14.

  72. Cheng X, Zhang T, Jia S, Xu B. Meta neurons improve spiking neural networks for efficient spatio-temporal learning. Neurocomputing. 2023;531:217–25.

    MATH  Google Scholar 

  73. Xie X, Yu B, Liu G, Zhan Q, Tang H. Effective active learning method for spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems.

  74. Wu Y, Deng L, Li G, Zhu J, Shi L. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience. 2018;12:331.

    MATH  Google Scholar 

  75. Liu F, Zhao W, Chen Y, Wang Z, Dai F. DynSNN: a dynamic approach to reduce redundancy in spiking neural networks. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 2130–2134.

  76. Yha B, Xh A, Meng DA, Bo X. A biologically plausible supervised learning method for spiking neural networks using the symmetric STDP rule. Neural Networks. 2020;121:387–95.

    Google Scholar 

  77. Ikegawa S-I, Saiin R, Sawada Y, Natori N. Rethinking the role of normalization and residual blocks for spiking neural networks. Sensors. 2022;22(8):2876.

    MATH  Google Scholar 

  78. Fang H, Mei Z, Shrestha A, Zhao Z, Li Y, Encoding Qiu Q, model, and architecture: Systematic optimization for spiking neural network in fpgas. In,. IEEE/ACM international conference on computer aided design (ICCAD). IEEE. 2020;2020:1–9.

  79. Zhang W, Li P. Spike-train level backpropagation for training deep recurrent spiking neural networks, Advances in neural information processing systems 32.

  80. Xiao M, Meng Q, Zhang Z, Wang Y, Lin Z. Training feedback spiking neural networks by implicit differentiation on the equilibrium state. Advances in Neural Information Processing Systems. 2021;34:14516–28.

    MATH  Google Scholar 

  81. Le Nguyen H, Chu D, Incremental neural synthesis for spiking neural networks. In,. IEEE symposium series on computational intelligence (SSCI). IEEE. 2022;2022:649–56.

  82. Lan M, Xiong X, Jiang Z, Lou Y. PC-SNN: supervised learning with local Hebbian synaptic plasticity based on predictive coding in spiking neural networks, arXiv:2211.15386.

  83. Wu X, Wang Y, Tang H, Yan R. A structure-time parallel implementation of spike-based deep learning. Neural Networks. 2019;113:72–8.

    MATH  Google Scholar 

  84. Wang T, Wang H, He J, Zhong Z, Tang F, Zhou X, Yu S-M, Liu L, Wu N, Tian M, Morphbungee: An edge neuromorphic chip for high-accuracy on-chip learning of multiple-layer spiking neural networks, in, et al. IEEE biomedical circuits and systems conference (BioCAS). IEEE. 2022;2022:255–9.

  85. Mozafari M, Kheradpisheh SR, Masquelier T, Nowzari-Dalini A, Ganjtabesh M. First-spike-based visual categorization using reward-modulated STDP. IEEE transactions on neural networks and learning systems. 2018;29(12):6178–90.

Download references

Funding

This work was supported by the National Key R&D Program of China under Grant 2018AAA0100300, and the National Natural Science Foundation of China under Grant 62172073, 62076182, 62176040.

Author information

Authors and Affiliations

  1. School of Science, Dalian Minzu University, Dalian, 116600, China

    Fang Liu

  2. School of Mathematical Sciences, Dalian University of Technology, Dalian, 116024, China

    Jialin Xu, Jie Yang & Wei Wu

  3. Key Laboratory for Computational Mathematics and Data Intelligence of Liaoning Province, Dalian, 116024, China

    Jialin Xu, Jie Yang & Wei Wu

Authors
  1. Fang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jie Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Yang.

Ethics declarations

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Conflict of Interest

The authors declare no competing interests.

Open Access

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Fang Liu and Jialin Xu are co-first authors of this manuscript.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, F., Xu, J., Yang, J. et al. A Novel Multi-Type Image Coding Method Acting on Supervised Hierarchical Deep Spiking Convolutional Neural Networks for Image Classification. Cogn Comput 17, 9 (2025). https://doi.org/10.1007/s12559-024-10355-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12559-024-10355-4

Keywords