Skip to main content
SpringerLink
Log in

A Scale and Translation Invariant Approach for Early Classification of Spatio-Temporal Patterns Using Spiking Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

This paper addresses the problem of encoding and classifying spatio-temporal patterns, which are typical for human actions or gestures. The proposed method has the following main contributions: (i) it requires a very small number of training examples, (ii) it accepts variable sized input patterns, (iii) it is invariant to scale and translation, and (iv) it enables early recognition, from only partial information of the pattern. The underlying representation employed is a spiking neural network with axonal conductance delay. We designed a novel approach for mapping spatio-temporal patterns to spike trains, which are used to stimulate the network. The pattern features emerge in the network as a result of this stimulation in the form of polychronous neuronal groups, which are used for classification. The proposed method is validated on a set of gestures representing the digits from \(0\) to \(9\), extracted from video data of a human drawing the corresponding digits. The paper presents a comparison with several other standard pattern recognition approaches. The results show that the proposed approach significantly outperforms these methods, it is invariant to scale and translation, and it has the ability to recognize patterns from only partial information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Arulampalam MS, Maskell S, Gordon N, Clapp T (2002) A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Trans Signal Process 50(2):174–188

    Article  Google Scholar 

  2. Bahlmann C, Burkhardt H (2004) The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping. IEEE Trans Pattern Anal Mach Intel 26(3):299–310

    Article  Google Scholar 

  3. Beyeler M, Dutt ND, Krichmar JL (2013) Categorization and decision-making in a neurobiologically plausible spiking network using a stdp-like learning rule. Neural Netw 48:109–124

    Article  Google Scholar 

  4. Bobick AF, Wilson AD (1997) A state-based approach to the representation and recognition of gesture. IEEE Trans Pattern Anal Mach Intel 19(12):1325–1337

    Article  Google Scholar 

  5. Carreira-Perpinan MA, Hinton GE (2005) On contrastive divergence learning. In: Proceedings of the tenth international workshop on artificial intelligence and statistics, Citeseer, pp 33–40

  6. Chang C-C, Lin C-J (2011) Libsvm: a library for support vector machines. ACM Trans Intel Syst Technol 2(3):27

    Article  Google Scholar 

  7. Ciresan DC, Meier U, Gambardella LM, Schmidhuber J (2010) Deep, big, simple neural nets for handwritten digit recognition. Neural Comput 22(12):3207–3220

    Article  Google Scholar 

  8. Davis J, Shah M (1994) Visual gesture recognition. In: IEEE proceedings of vision, image and signal processing, IET, vol 141, pp 101–106

  9. Djioua M, Plamondon R (2009) A new algorithm and system for the characterization of handwriting strokes with delta-lognormal parameters. IEEE Trans Pattern Anal Mach Intel 31(11):2060–2072

    Article  Google Scholar 

  10. Doucet A, De Freitas N, Gordon N (2001) Sequential Monte Carlo methods in practice. Springer, New York

    Book  MATH  Google Scholar 

  11. Duda PE, Richard O (1973) Hart, pattern classification and scene analysis. Wiley-Interscience, Oxford

    MATH  Google Scholar 

  12. Gonzalez RC, Woods RE (2002) Digital image processing. Gatesmark Publishing, Knoxville

    Google Scholar 

  13. Graves A, Liwicki M, Fernández S, Bertolami R, Bunke H, Schmidhuber J (2009) A novel connectionist system for unconstrained handwriting recognition. IEEE Trans Pattern Anal Mach Intel 31(5):855–868

    Article  Google Scholar 

  14. Haykin S, Network N (2004) A comprehensive foundation. Neural Netw 2:2004

    Google Scholar 

  15. Hinton G, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  16. Hong P, Turk M, Huang TS (2000) Gesture modeling and recognition using finite state machines. In: Proceedings of fourth IEEE international conference on automatic face and gesture recognition, pp 410–415

  17. Hu J, Gek Lim S, Brown MK (2000) Writer independent on-line handwriting recognition using an HMM approach. Pattern Recog 33(1):133–147

    Article  Google Scholar 

  18. Isard M, Blake A (1996) Contour tracking by stochastic propagation of conditional density. In: Computer visionECCV’96. Springer, New York, pp 343–356

  19. Isard M, Blake A (1998) Condensationconditional density propagation for visual tracking. Int J Comput Vis 29(1):5–28

    Article  Google Scholar 

  20. Izhikevich EM (2006) Polychronization: computation with spikes. Neural Comput 18(2):245–282

    Article  MathSciNet  MATH  Google Scholar 

  21. Izhikevich EM et al (2003) Simple model of spiking neurons. IEEE Trans Neural Netw 14(6):1569–1572

    Article  MathSciNet  Google Scholar 

  22. Izhikevich EM, Gally JA, Edelman GM (2004) Spike-timing dynamics of neuronal groups. Cereb Cortex 14(8):933–944

    Article  Google Scholar 

  23. Jaccard P (1912) The distribution of the flora in the alpine zone. 1. New Phytol 11(2):37–50

    Article  Google Scholar 

  24. Kandel ER, Schwartz JH, Jessell TM et al (2000) Principles of neural science, vol 4. McGraw-Hill, New York

  25. Karimpouli S, Fathianpour N, Roohi J (2010) A new approach to improve neural networks’ algorithm in permeability prediction of petroleum reservoirs using supervised committee machine neural network (scmnn). J Pet Sci Eng 73(3):227–232

    Article  Google Scholar 

  26. Kass M, Witkin A, Terzopoulos D (1988) Snakes: active contour models. Int J Comput Vis 1(4):321–331

    Article  MATH  Google Scholar 

  27. Kwok C, Fox D, Meila M (2004) Real-time particle filters. Proc IEEE 92(3):469–484

    Article  Google Scholar 

  28. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  29. Liwicki M, Bunke H, et al. (2006) Hmm-based on-line recognition of handwritten whiteboard notes. In: Tenth international workshop on frontiers in handwriting recognition

  30. Marquardt DW (1963) An algorithm for least-squares estimation of nonlinear parameters. J Soc Ind Appl Math 11(2):431–441

    Article  MathSciNet  MATH  Google Scholar 

  31. Murphy KP (2012) Machine learning: a probabilistic perspective. MIT Press, Cambridge

    MATH  Google Scholar 

  32. Palm RB (2012) Prediction as a candidate for learning deep hierarchical models of data. Technical University of Denmark, Palm

    Google Scholar 

  33. Paugam-Moisy H, Martinez R, Bengio S (2008) Delay learning and polychronization for reservoir computing. Neurocomputing 71(7):1143–1158

    Article  Google Scholar 

  34. Rabiner L (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  35. Ramamoorthy A, Vaswani N, Chaudhury S, Banerjee S (2003) Recognition of dynamic hand gestures. Pattern Recog 36(9):2069–2081

    Article  MATH  Google Scholar 

  36. Rekabdar B, Joorabian M, Shadgar B (2012) Artificial neural network ensemble approach for creating a negotiation model with ethical artificial agents. In: Artificial intelligence and soft computing. Springer, New York, pp 493–501

  37. Rekabdar B, Nicolescu M, Kelley R, Nicolescu M (2014) Unsupervised learning of spatio-temporal patterns using spike timing dependent plasticity. In: Artificial general intelligence. Springer, New York, pp 254–257

  38. Rekabdar B, Nicolescu M, Kelley R, Nicolescu M (2015) An unsupervised approach to learning and early detection of spatio-temporal patterns using spiking neural networks. J Intel Robot Syst 77(1):1–15. doi:10.1007/s10846-015-0179-1

  39. Samaria F, Young S (1994) Hmm-based architecture for face identification. Image Vis Comput 12(8):537–543

    Article  Google Scholar 

  40. Tao X, Michel HE (2004) Data clustering via spiking neural networks through spike timing-dependent plasticity. In: IC-AI, pp 168–173

  41. Welch G, Bishop G (1995) An introduction to the Kalman filter. Technical Report TR 95-041, Department of Computer Science, University of North Carolina

  42. Yamato J, Ohya J, Ishii K (1992) Recognizing human action in time-sequential images using hidden markov model. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition CVPR’92, pp 379–385

  43. Yang M-H, Ahuja N (2001) Recognizing hand gestures using motion trajectories. In: Face detection and gesture recognition for human–computer interaction. Springer, New York, pp 53–81

  44. Yeasin M, Chaudhuri S (2000) Visual understanding of dynamic hand gestures. Pattern Recog 33(11):1805–1817

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank the anonymous reviewers for their useful comments on improving our manuscript. This work has been supported by ONR Grant #N000141210860.

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, University of Nevada, Reno (UNR), Reno, USA

    Banafsheh Rekabdar, Monica Nicolescu, Mircea Nicolescu, Mohammad Taghi Saffar & Richard Kelley

Authors
  1. Banafsheh Rekabdar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Monica Nicolescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mircea Nicolescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammad Taghi Saffar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Kelley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Banafsheh Rekabdar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rekabdar, B., Nicolescu, M., Nicolescu, M. et al. A Scale and Translation Invariant Approach for Early Classification of Spatio-Temporal Patterns Using Spiking Neural Networks. Neural Process Lett 43, 327–343 (2016). https://doi.org/10.1007/s11063-015-9436-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-015-9436-3

Keywords

Search

Navigation