Skip to main content

Advertisement

SpringerLink
Log in

A comparative study on spiking neural network encoding schema: implemented with cloud computing

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Spiking neural networks (SNN) represents the third generation of neural network models, it differs significantly from the early neural network generation. The time is becoming the most important input. The presence and precise timing of spikes encapsulate have a meaning such as human brain behavior. However, deferent techniques are therefore required to submit a stimulus to the neural network to build the timing spike. The characteristics of these spikes are based on their firing time because of the stereotypical nature of the human brain. Neural networks (NN) as engineering tools Operate on analog quantities (analog input, analog output), SNN More powerful than classic NN Interesting to implement in hardware. But the Problem that is internally work with spike trains unequal analog signal, so this algorithm design to firstly convert analog function into spike trains which calling encoding (E) then Convert spike trains into analog function: which calling decoding (D), so to use spiking NN as engineering tool: communication problem must be solved using some international encoding algorithms. This paper discusses techniques of transforming data into a suitable form for SNN submission. We present a comparative study on SNN encoding schema that effect on SNN performance in hardware and software implementation, however, this is the first comprehensive study to discuss encoding algorithms in SNNs in details, which involved the advantages, disadvantages and when and where we can use and implements the encoding algorithms, with focusing on some examples implement SNN in cloud computing generally, and which algorithms still unused in the world of cloud computing to make the door open for new researcher.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Adapted from Lichtsteiner and Delbruck [66])

Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Maass, W., Bishop, C.M.: Pulsed Neural Networks. MIT, Cambridge (2001)

    MATH  Google Scholar 

  2. Andrew, A.M.: Spiking neuron models: single neurons, populations, plasticity. Kybernetes 100, 100 (2003). https://doi.org/10.1108/k.2003.06732gae.003

    Article  Google Scholar 

  3. Gerstner, W.: Time structure of the activity in neural network models. Phys. Rev. E 51(1), 738 (1995)

    Article  MathSciNet  Google Scholar 

  4. Gerstner, W., Kistler, W.: Spiking Neuron Models. Cambridge University Press, Cambridge (2002)

    Book  MATH  Google Scholar 

  5. Kistler, W.M., Gerstner, W., van Hemmen, J.L.: Reduction of the Hodgkin–Huxley equations to a single-variable threshold model. Neural Comput. 9(5), 1015–1045 (1997)

    Article  Google Scholar 

  6. Hodgkin, A.L., Huxley, A.F.: Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo. J. Physiol. 116(4), 449–472 (1952)

    Article  Google Scholar 

  7. Izhikevich, E.M.: Which model to use for cortical spiking neurons? IEEE Trans. Neural Netw. 15(5), 1063–1070 (2004)

    Article  Google Scholar 

  8. Izhikevich, E.M., Moehlis, J.: Dynamical systems in neuroscience: the geometry of excitability and bursting. SIAM Rev. 50(2), 397 (2008)

    Google Scholar 

  9. Hamed, H.N.A., Kasabov, N., Shamsuddin, S.M.: Probabilistic evolving spiking neural network optimization using dynamic quantum-inspired particle swarm optimization. Aust. J. Intell. Inf. Process. Syst. 11(1), 23–28 (2010)

    Google Scholar 

  10. Schliebs, S., Defoin-Platel, M., Worner, S., Kasabov, N.: Integrated feature and parameter optimization for an evolving spiking neural network: exploring heterogeneous probabilistic models. Neural Netw. 22(5), 623–632 (2009)

    Article  Google Scholar 

  11. Kandias, M., Virvilis, N., Gritzalis, D.: The insider threat in cloud computing. In: International Workshop on Critical Information Infrastructures Security, pp. 93–103. Springer, Berlin (2011)

  12. Almomani, A., Alauthman, M., Albalas, F., Dorgham, O., Obeidat, A.: An online intrusion detection system to cloud computing based on NeuCube algorithms. Int. J. Cloud Appl. Comput. (IJCAC) 8(2), 96–112 (2018)

    Google Scholar 

  13. Chadha, A., Abbas, A., Andreopoulos, Y.: Video Classification with CNNs: Using the Codec as a Spatio-Temporal Activity Sensor. arXiv preprint. arXiv:1710.05112 (2017)

  14. Martinelli, E., D’Amico, A., Di Natale, C.: Spike encoding of artificial olfactory sensor signals. Sensors Actuators B 119(1), 234–238 (2006)

    Article  Google Scholar 

  15. Loiselle, S., Rouat, J., Pressnitzer, D., Thorpe, S.: Exploration of rank order coding with spiking neural networks for speech recognition. In: Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN’05), pp. 2076–2080. IEEE, Montreal (2005)

  16. Eurich, C.W., Wilke, S.D.: Multidimensional encoding strategy of spiking neurons. Neural Comput. 12(7), 1519–1529 (2000)

    Article  Google Scholar 

  17. Van Rullen, R., Thorpe, S.J.: Neural Comput. Neural Comput. 13(6), 1255–1283 (2001)

    Article  Google Scholar 

  18. Hopfield, J.J.: Pattern recognition computation using action potential timing for stimulus representation. Nature 376(6535), 33–36 (1995)

    Article  Google Scholar 

  19. Izhikevich, E.M.: Simple model of spiking neurons. IEEE Trans Neural Netw 14(6), 1569–1572 (2003)

    Article  MathSciNet  Google Scholar 

  20. Maass, W.: Computing with spiking neurons. In: Maass, W., Bishop, C.M. (eds.) Pulsed Neural Networks, pp. 55–85. MIT, Cambridge (1999)

    Google Scholar 

  21. Belatreche, A., Maguire, L.P., McGinnity, M.: Advances in design and application of spiking neural networks. Soft. Comput. 11(3), 239–248 (2007)

    Article  Google Scholar 

  22. Brody, C.D., Hopfield, J.: Simple networks for spike-timing based computation. Neuron 37, 843–852 (2003)

    Article  Google Scholar 

  23. Booij, O., tat Nguyen, H.: A gradient descent rule for spiking neurons emitting multiple spikes. Inf. Process. Lett. 95(6), 552–558 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  24. Bohte, S.M., La Poutré, H., Kok, J.N.: Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks. IEEE Trans. Neural Netw. 13(2), 426–435 (2002)

    Article  Google Scholar 

  25. Maguire, L.P., McGinnity, T.M., Glackin, B., Ghani, A., Belatreche, A., Harkin, J.: Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing 71(1), 13–29 (2007)

    Article  Google Scholar 

  26. Zuppicich, A., Soltic, S.: FPGA implementation of an evolving spiking neural network. In: International Conference on Neural Information Processing, pp. 1129–1136. Springer, Berlin (2008)

  27. Dhoble, K.: Spatio-/spectro-temporal pattern recognition using evolving probabilistic spiking neural networks. Auckland University of Technology, Auckland (2013)

    Google Scholar 

  28. Dayan, P., Abbott, L.F.: Theoretical Neuroscience, vol. 806. MIT, Cambridge (2001)

    MATH  Google Scholar 

  29. Gabbiani, F., Metzner, W.: Encoding and processing of sensory information in neuronal spike trains. J. Exp. Biol. 202(10), 1267–1279 (1999)

    Google Scholar 

  30. Gabbiani, F.: Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons. Netw. Comput. Neural Syst. 7(1), 61–85 (1996)

    MATH  Google Scholar 

  31. Koch, C.: Linear stimulus encoding and decoding (1999). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.42.3344&rep=rep1&type=pdf

  32. Schrauwen, B., Van Campenhout, J.: BSA, a fast and accurate spike train encoding scheme. In: Proceedings of the International Joint Conference on Neural Networks, pp. 2825–2830. IEEE, Piscataway (2003)

  33. Gerstner, W., Kistler, W.M.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge (2002)

    Book  MATH  Google Scholar 

  34. Gerstner, W.: What is different with spiking neurons? In: Mastebroek, H.A.K., Vos, J.E. (eds.) Plausible Neural Networks for Biological Modelling, pp. 23–48. Springer, Dordrecht (2001)

    Chapter  Google Scholar 

  35. Yu, Q., Tan, K.C., Tang, H.: Pattern recognition computation in a spiking neural network with temporal encoding and learning. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE, Granada (2012)

  36. Gross, C.G.: Genealogy of the “grandmother cell”. Neuroscientist 8(5), 512–518 (2002)

    Article  Google Scholar 

  37. Donachy, S.: Spiking neural networks: neuron models, plasticity, and graph applications. Thesis (2015). Available at https://scholarscompass.vcu.edu/cgi/viewcontent.cgi?article=5025&context=etd

  38. Thorpe, S.J.: Grandmother cells and distributed representations. In: Visual Population Codes: Toward a Common Multivariate Framework for Cell Recording and Functional Imaging, pp. 23–51. MIT, Cambridge (2011)

  39. Cruz, B., Gupta, D., Kapoor, A., Haifei, L., McLean, D., Moreno, F.: McAfee Labs Threats Report. McAfee, Santa Clara (2016)

    Google Scholar 

  40. Meftah, B., Lézoray, O., Chaturvedi, S., Khurshid, A.A., Benyettou, A.: Image processing with spiking neuron networks. In: Yang, X.-S. (ed.) Artificial Intelligence, Evolutionary Computing and Metaheuristics: In the Footsteps of Alan Turing, pp. 525–544. Springer, Berlin (2013)

    Chapter  Google Scholar 

  41. Szatmáry, B., Izhikevich, E.M.: Spike-timing theory of working memory. PLoS Comput. Biol. 6(8), e1000879 (2010)

    Article  MathSciNet  Google Scholar 

  42. Kiselev, M.: Rate coding vs. temporal coding-is optimum between? In: IEEE 2016 International Joint Conference on Neural Networks (IJCNN), pp. 1355–1359 (2016)

  43. Dhilipan, A., Preethi, J., Sreeshakthy, M., Sangeetha, V.: A survey on pattern recognition using spiking neural networks with temporal encoding and learning. Int. J. Res. Advent Technol. 2(11), 121–125 (2014)

    Google Scholar 

  44. Dayan, P., Abbott, L.: Theoretical neuroscience: computational and mathematical modeling of neural systems. J. Cogn. Neurosci. 15(1), 154–155 (2003)

    Article  Google Scholar 

  45. Du, D., Odame, K.: An energy-efficient spike encoding circuit for speech edge detection. Analog Integr. Circuits Signal Process. 75(3), 447–458 (2013)

    Article  Google Scholar 

  46. Martens, M.B., Houweling, A.R., Tiesinga, P.H.: Anti-correlations in the degree distribution increase stimulus detection performance in noisy spiking neural networks. J. Comput. Neurosci. 42(1), 87–106 (2017)

    Article  Google Scholar 

  47. Muhammad, C.: Neuromodulation based control of autonomous robots on a cloud computing platform. Electron. Theses Diss. (2014). Available at https://digitalcommons.georgiasouthern.edu/etd/1203

  48. Paugam-Moisy, H., Bohte, S.: Computing with spiking neuron networks. In: Handbook of Natural Computing, pp. 335–376. Springer, Heidelberg (2012)

  49. Yu, Q., Tang, H., Hu, J., Tan, K.C.: Rapid feedforward computation by temporal encoding and learning with spiking neurons. In: Neuromorphic Cognitive Systems, pp. 19–41. Springer, Berlin (2017)

  50. Gardner, B., Grüning, A.: Supervised learning in spiking neural networks for precise temporal encoding. PLoS ONE 11(8), e0161335 (2016)

    Article  Google Scholar 

  51. Ahn, S., Lee, B., Kim, M.: A novel fast CU encoding scheme based on spatiotemporal encoding parameters for HEVC inter coding. IEEE Trans. Circuits Syst. Video Technol. 25(3), 422–435 (2015)

    Article  Google Scholar 

  52. Thorpe, S., Gautrais, J.: Rank Order Coding. In: Bower, J.M. (ed.) Computational Neuroscience: Trends in Research, pp. 113–118. Springer, Boston (1998)

    Chapter  Google Scholar 

  53. Sen Bhattacharya, B., Furber, S.: Information recovery from rank-order encoded images. Workshop in School of Computer Science, University of Manchester, Manchester, UK (2006). Available at http://eprints.lincoln.ac.uk/10602/1/workshop_surrey_aug06.pdf

  54. Delbruck, T., Lichtsteiner, P.: Fast sensory motor control based on event-based hybrid neuromorphic-procedural system. In: IEEE International Symposium on Circuits and Systems (ISCAS 2007), pp. 845–848. IEEE, Lausanne (2007)

  55. Delorme, A., Perrinet, L., Thorpe, S.J.: Networks of integrate-and-fire neurons using Rank Order Coding B: spike timing dependent plasticity and emergence of orientation selectivity. Neurocomputing 38, 539–545 (2001)

    Article  Google Scholar 

  56. Delorme, A., Thorpe, S.J.: Face identification using one spike per neuron: resistance to image degradations. Neural Netw. 14(6), 795–803 (2001)

    Article  Google Scholar 

  57. Wysoski, S.G., Benuskova, L., Kasabov, N.: Evolving spiking neural networks for audiovisual information processing. Neural Netw. 23(7), 819–835 (2010)

    Article  Google Scholar 

  58. Thangamalar, C., Elakkani, M., Mekala, V.: Secure ranked multi keyword hierarchical search arrangement over encoded cloud data. Int. J. Eng. Tech. 3(6), 504–506 (2017)

    Google Scholar 

  59. Hough, M., De Garis, H., Korkin, M., Gers, F., Nawa, N.E.: Spiker: Analog waveform to digital spiketrain conversion in ATR’s artificial brain (cam-brain) project. In: International Conference on Robotics and Artificial Life (1999)

  60. Korkin, M., Fehr, G., Jeffery, G.: Evolving hardware on a large scale. In: IEEE Proceedings of the Second NASA/DoD Workshop on Evolvable Hardware, pp. 173–181 (2000)

  61. Schrauwen, B., Campenhout, J.V.: BSA, a fast and accurate spike train encoding scheme. In: Proceedings of the International Joint Conference on Neural Networks, vol. 2824, 20–24 July 2003, pp. 2825–2830 (2003)

  62. Valadez, S., Sossa, H., Santiago-Montero, R., Guevara, E.: Encoding polysomnographic signals into spike firing rate for sleep staging. In: Mexican Conference on Pattern Recognition, pp. 282–291. Springer, Cham (2015)

  63. Wikipedia: Blausen 0657 MultipolarNeuron.png. https://en.wikipedia.org/wiki/File:Blausen_0657_MultipolarNeuron.png (2017)

  64. Yu, Q., Tang, H., Tan, K.C., Yu, H.: A brain-inspired spiking neural network model with temporal encoding and learning. Neurocomputing 138, 3–13 (2014)

    Article  Google Scholar 

  65. Tait, A.N., Nahmias, M.A., Tian, Y., Shastri, B.J., Prucnal, P.R.: Photonic neuromorphic signal processing and computing. In: Naruse, M. (ed.) Nanophotonic Information Physics. Springer, Berlin (2014)

    Google Scholar 

  66. Lichtsteiner, P., Delbruck, T.: A 64 × 64 AER logarithmic temporal derivative silicon retina. In: Research in Microelectronics and Electronics, 2005 PhD, pp. 202–205. IEEE (2005)

  67. Sugase, Y., Shigeru, Y., Shoogo, U., Kawano, K.: Biological sciences 300/301, Smith college | neurophysiology, case 6: signaling by a face-selective neuron (2017). Available at http://www.science.smith.edu/departments/neurosci/courses/bio330/cases/case6-face.html

  68. Bouton, C.: Cracking the neural code, treating paralysis and the future of bioelectronic medicine. JIM J. Intern. Med. 282(1), 37–45 (2017). https://doi.org/10.1111/joim.12610

    Article  Google Scholar 

  69. van der Meer, M.A., Carey, A.A., Tanaka, Y.: Optimizing for generalization in the decoding of internally generated activity in the hippocampus. Hippocampus 27(5), 580–595 (2017)

    Article  Google Scholar 

  70. Ray, S., Heinen, S.J.: A mechanism for decision rule discrimination by supplementary eye field neurons. Exp. Brain Res. 233(2), 459–476 (2015)

    Article  Google Scholar 

  71. Torikai, H., Nishigami, T.: A novel chaotic spiking neuron and its paralleled spike encoding function. In: 2009 International Joint Conference on Neural Networks (IJCNN 2009), pp. 3132–3139. IEEE

  72. Alien, R.A.: Q. Disadvantages of distributed representations. https://www.reddit.com/r/MachineLearning/comments/4e50kh/q_disadvantages_of_distributed_representations/ (2017)

  73. Kasabov, N., Dhoble, K., Nuntalid, N., Indiveri, G.: Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. Neural Netw. 41, 188–201 (2013). https://doi.org/10.1016/j.neunet.2012.11.014

    Article  Google Scholar 

  74. Kasabov, N., Scott, N.M., Tu, E., Marks, S., Sengupta, N., Capecci, E., Othman, M., Doborjeh, M.G., Murli, N., Hartono, R.: Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: design methodology and selected applications. Neural Netw. 78, 1–14 (2016)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by Al-Balqa Applied University, Al-Huson University College, Department of Information Technology, 50, Irbid, Jordan.

Author information

Authors and Affiliations

  1. IT-Department, Al-Huson University College, Al-Balqa Applied University, P. O. Box 50, Irbid, Jordan

    Ammar Almomani

  2. Department of Computer Science, Faculty of Information Technology, Zarqa University, Zarqa, Jordan

    Mohammad Alauthman

  3. Prince Abdullah Ben Ghazi Faculty of Information Technology, Al-Balqa Applied University, Al Salt, 19117, Jordan

    Mohammed Alweshah & O. Dorgham

  4. Department of Computer Science, Jordan University of Science and Technology, P.O. Box 3030, Irbid, Jordan

    Firas Albalas

Authors
  1. Ammar Almomani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Alauthman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Alweshah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. O. Dorgham
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Firas Albalas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ammar Almomani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Almomani, A., Alauthman, M., Alweshah, M. et al. A comparative study on spiking neural network encoding schema: implemented with cloud computing. Cluster Comput 22, 419–433 (2019). https://doi.org/10.1007/s10586-018-02891-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-018-02891-0

Keywords

Search

Navigation