Skip to main content
SpringerLink
Log in

Principal motion components for one-shot gesture recognition

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

This paper introduces principal motion components (PMC), a new method for one-shot gesture recognition. In the considered scenario a single training video is available for each gesture to be recognized, which limits the application of traditional techniques (e.g., HMMs). In PMC, a 2D map of motion energy is obtained per each pair of consecutive frames in a video. Motion maps associated to a video are processed to obtain a PCA model, which is used for recognition under a reconstruction-error approach. The main benefits of the proposed approach are its simplicity, easiness of implementation, competitive performance and efficiency. We report experimental results in one-shot gesture recognition using the ChaLearn Gesture Dataset; a benchmark comprising more than 50,000 gestures, recorded as both RGB and depth video with a Kinect™camera. Results obtained with PMC are competitive with alternative methods proposed for the same data set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We assume each video to be processed depicts a single gesture. Gesture segmentation is an open problem by itself that we do not approach in this paper, although we evaluate the performance of our method using gestures manually and automatically segmented with a basic technique.

  2. http://gesture.chalearn.org/.

  3. Experiments were performed in a workstation with Intel® Core™i7-2600 CPU at 3.4 GHz and 8GB in RAM.

  4. http://www.kaggle.com/c/GestureChallenge/leaderboard.

  5. http://www.kaggle.com/c/GestureChallenge2/leaderboard.

References

  1. Aggarwal JK, Cai Q (1999) Human motion analysis: a review. Comput Vis Image Underst 73(3):428–440

    Article  Google Scholar 

  2. Aggarwal JK, Ryoo MS (2011) Human activity analysis: a review. ACM Comput Surveys 43(3):Atr. 16

  3. Agrawal T, Chaudhuri S (2003) Gesture recognition using motion histogram. In: Proceedings of the Indian National Conference of Communications, pp 438–442

  4. Bekios-Calfa J, Buenaposada JM, Baumela L (2011) Class-conditional probabilistic principal component analysis: application to gender recognition. Computacion y Sistemas 14(4):383–391

    Google Scholar 

  5. Bishop CM (2006) Pattern Recognit Mach Learn. Springer, New York

    Google Scholar 

  6. Bobick AF, Davis JW (2001) The recognition of human movement using temporal templates. IEEE Trans Pattern Anal Mach Intell 23(3):257–267

    Article  Google Scholar 

  7. Chaudhry R, Ravichandran A, Hager G, Vidal R (2009) Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In: Proceedings of the IEEE conference on pattern recognition and computer vision. IEEE, pp 1651–1657

  8. Cheemaa MS, Eweiwia A, Bauckhage C (2014) Human activity recognition by separating style and content. Pattern Recognit Lett 50(1):130–138

    Article  Google Scholar 

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE conference on pattern recognition and computer vision, pp 886–893

  10. Davis JW (2001) Hierarchical motion history images for recognizing human motion. In: Proceedings of the IEEE workshop on detection and recognition of events in video, pp 39–46

  11. Eichner M, Marín-Jiménez MJ, Ferrari V, Zisserman A (2012) 2d articulated human pose estimation and retrieval in (almost) unconstrained still images. Int J Comput Vision 99(2):190–214

    Article  MathSciNet  Google Scholar 

  12. Escalera S, Baró X, Gonzalez J, Bautista MA, Madadi M, Reyes M, Ponce V, Escalante HJ, Shotton J, Guyon I (2015) ChaLearn looking at people challenge 2014: dataset and results. In: Proceedings of ECCV 2014 workshops, LNCS, vol 8925. Springer, pp 459–473

  13. Escalera S, Gonzalez J, Baró X, Reyes M, Guyon I, Athistos V, Escalante HJ, Sigal L, Argyros A, Sminchisescu C, Sclaroff S (2013) ChaLearn multi-modal gesture recognition 2013: grand challenge and workshop summary. In: ICMI ’13 Proceedings of the 15th ACM on international conference on multimodal interaction. ACM, pp 365–368

  14. Freeman WT, Roth M (1995) Orientation histograms for hand gesture recognition. In: Proceedings of the IEEE international workshop on automatic face and gesture recognition

  15. Gomez JC, Moens MF (2012) PCA document reconstruction for email classification. Comput Stat Data Anal 56:741–751

    Article  MathSciNet  Google Scholar 

  16. Guyon I, Athitsos V, Jangyodsuk P, Escalante HJ (2014) The chalearn gesture dataset (cgd 2011). Mach Vis Appl 25(8):1929–1951

    Article  Google Scholar 

  17. Guyon I, Athitsos V, Jangyodsuk P, Escalante HJ, Hamner B (2013) Results and analysis of the ChaLearn gesture challenge 2012. In: WDIA: advances in depth image analysis and applications, LNCS, vol 7854. Springer, pp 186–204

  18. Guyon I, Athitsos V, Jangyodsuk P, Hammer B, Escalante HJ (2012) ChaLearn gesture challenge: design and first results. In: Proceedings of the conference on computer vision and pattern recognition worshops, Rhode Island. IEEE, pp 1–6

  19. Guyon I, Gunn S, Nikravesh M, Zadeh L (eds) Feature extraction, foundations and applications, Studies in fuzzines and soft computing, vol 207. Springer, Berlin

  20. Hoffmann H (2007) Kernel pca for novelty detection. Pattern Recognit 40(3):863–874

    Article  MATH  Google Scholar 

  21. Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, New York

    MATH  Google Scholar 

  22. Lee HK, Kim JH (1999) An hmm-based threshold model approach for gesture recognition. IEEE Trans Pattern Anal Mach Intell 21(10):961–973

    Article  Google Scholar 

  23. Liu W, Tao D, Wang Y, Lu K (2015) Multiview hessian regularized logistic regression. Signal Process 110:101–107

    Article  Google Scholar 

  24. Liu W, Li Y, Lin X, Tao D, Wang Y (2014) Hessian-regularized co-training for social activity recognition. PLoS ONE 9(9):1–10

    Google Scholar 

  25. Liu W, Zhang H, Tao D, Wang Y, Lu K (2015) Large-scale paralleled sparse principal component analysis. Multimedia Tools Appl

  26. Liu YM (2012) Human gesture recognition on product manifolds. J Mach Learn Res 13(2012):3297–3321

    MathSciNet  MATH  Google Scholar 

  27. Luo A, Kong X, Zeng G, Fan J (2010) Human action detection via boosted local motion histograms. Mach Vis Appl 21(3):377–389

    Article  Google Scholar 

  28. Mahbub U, Imtiaz H, Roy T, Rahman S, Rahman-Ahad MA (2013) A template matching approach to one-shot-learning gesture recognition. Pattern Recognit Lett 34(15):1780–1788

    Article  Google Scholar 

  29. Mahbub U, Roy T, Rahman S, Imtiaz H, Serikawa S, Rahman-Ahad MA (2013) A template matching approach to one-shot-learning gesture recognition. In: Proceedings of the 1st international conference on industrial application engineering, pp 186–193

  30. Malagon-Borja L, Fuentes O (2009) Object detection using image reconstruction with pca. Image Vis Comput 27(1–2):2–9

    Article  Google Scholar 

  31. Malgireddy MR, Nwogu I, Govindaraju V (2013) Language-motivated approaches to action recognition. J Mach Learn Res 14:2189–2212

    MathSciNet  Google Scholar 

  32. Martin J, Crowley JL (1997) An appearance-based approach to gesture recognition. In: Proceedings of the 9th international conference on image analysis and processing, LNCS, vol 1311. Springer, pp 340–347

  33. Mitra S (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybern Part C Appl Rev 37(3):311–324

    Article  Google Scholar 

  34. Munoz-Salinas R, Medina-Carnicer R, Madrid-Cuevas FJ, Carmona Potayo A (2008) Histograms of optical flow for efficient representation of body motion. Pattern Recognit Lett 29:319–329

  35. Pers J, Sulic V, Kristan M, Perse M, Polanec K, Kovaviv S (2010) Histograms of optical flow for efficient representation of body motion. Pattern Recognit Lett 31:1369–1379

    Article  Google Scholar 

  36. Polana R, Nelson R (1994) Low level recognition of human motion. In: Proceedings of the IEEE workshop on motion of non-rigid and articulated objects, pp 77–82

  37. Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  38. Shao L, Ji L (2009) Motion histogram analysis based key frame extraction for human/activity representation. In: Proceedings of the Canadian conference on computer and robot vision, pp 88, 92. IEEE

  39. Tax D (2001) One-class classification. PhD thesis, Delft University of Technology

  40. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  41. Wan J, Ruan Q, Deng S, Li W (2013) One-shot learning gesture recognition from RGB-D data using bag of features. J Mach Learn Res 14:2549–2582

    Google Scholar 

  42. Wang H, Ullah MM, Klaser A, Laptev I, Schmid C (2009) Evaluation of local spatio-temporal features for action recognition. In: Proceedings of the British machine vision conference, pp 1–11

  43. Wu D, Zhu F, Shao L (2012) One-shot learning gesture recognition from rgbd images. In: Proceedings of the conference on computer vision and pattern recognition worshops, Rhode Island, pp 7–12. IEEE

  44. Wu S, Pan W, Jiang F, Gao Y, Zhao D (2012) A mutiple-layered gesture recognition system for one-shot learning. In: ICPR 2012 gesture recognition workshop

  45. Yi H, Rajan D, Chia LT (2005) A new motion histogram to index motion content in video segments. Pattern Recognit Lett 26:1221–1231

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Instituto Nacional de Astrfisica, Optica y Electronica, Luis Enrique Erro No. 1, Tonantzintla, Puebla, 72840, Mexico

    Hugo Jair Escalante

  2. ChaLearn, 955 Creston Road, Berkeley, CA, 94708, USA

    Hugo Jair Escalante & Isabelle Guyon

  3. Computer Science and Engineering Department, University of Texas at Arlington, 701 S. Nedderman Drive, Arlington, TX, 76019, USA

    Vassilis Athitsos & Pat Jangyodsuk

  4. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Jun Wan

Authors
  1. Hugo Jair Escalante
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isabelle Guyon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vassilis Athitsos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pat Jangyodsuk
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hugo Jair Escalante.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Escalante, H.J., Guyon, I., Athitsos, V. et al. Principal motion components for one-shot gesture recognition. Pattern Anal Applic 20, 167–182 (2017). https://doi.org/10.1007/s10044-015-0481-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-015-0481-3

Keywords

Search

Navigation