Skip to main content
SpringerLink
Log in

A Dynamic Gesture and Posture Recognition System

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents a real time dynamic hand gesture and posture recognition system based on a neural network and a Hidden Markov Model. For skin color segmentation an adaptive online trained skin color model is used, while the hand posture recognition is accomplished through a likelihood-based classification technique of geometric features. A novel trajectory smoothing technique based on Self Organized Neural Network is introduced to improve HMM classification performance of dynamic gestures. The aim of the proposed system is the creation of a visual dictionary combining hand postures and dynamic gestures. The system has been successfully tested with many people under varying light conditions and different web cameras.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Murthy, G., Jadon, R.: A review of vision based hand gestures recognition. Int. J. Inf. Technol. Knowl. Manag. 2(2), 405–410 (2009)

    Google Scholar 

  2. Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.-F., Kirbas, C., McCullough, K.E., Ansari, R.: Multimodal human discourse: gesture and speech. ACM Trans. Comput-Hum. Interact. (TOCHI). 9(3), 171–193 (2002)

    Article  Google Scholar 

  3. Van den Bergh, M., Van Gool, L.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV) (2011)

  4. Elmezain, M., Al-Hamadi, A., Appenrodt, J., Michaelis, B.: A hidden markov model-based continuous gesture recognition system for hand motion trajectory. In: 19th International Conference on Pattern Recognition, ICPR (2008)

  5. Kollorz, E., Penne, J., Hornegger, J., Barke, A.: Gesture recognition with a time-of-flight camera. Int. J. Intell. Syst. Technol. Appl. 5(3), 334–343 (2008)

    Google Scholar 

  6. Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin-color modeling and detection methods. Pattern Recognit. 40(3), 1106–1122 (2007)

    Article  MATH  Google Scholar 

  7. Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. Graphicon (2003)

  8. Doulamis, N., Doulamis, A., Kosmopoulos, D.: Content-based decomposition of gesture videos. In: IEEE Workshop on Signal Processing Systems Design and Implementation (2005)

  9. Araki, R., Gohshi, S., Ikenaga, T.: Real-time both hands tracking using CAMshift with motion mask and probability reduction by motion prediction. In: Signal & Information Processing Association Annual Summit and Conference, (APSIPA ASC) 2012. Asia-Pacific (2012)

  10. Oikonomidis, I., Kyriazis, N., Argyros, A.: Efficient model-based 3d tracking of hand articulations using kinect. In: British Machine Vision Conference (2011)

  11. Athitsos, V., Wang, H., Stefan, A.: A database-based framework for gesture recognition. Pers. Ubiquit. Comput. 14(6), 511–526 (2010)

    Article  Google Scholar 

  12. Malassiotis, S., Strintzis, M.: Real-time hand posture recognition using range data. Image Vis. Comput. 26(7), 1027–1037 (2008)

    Article  Google Scholar 

  13. Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: a review. Comp. Vision Image Underst. 108(1), 52–73 (2007)

    Article  Google Scholar 

  14. Hasan, M.M., Mishra, P.K.: Hand gesture modeling and recognition using geometric features: a review. Can. J. Image Process. Comput. Vis. 3(1), 12–26 (2012)

    Google Scholar 

  15. Mitra, S., Acharya, T.: Gesture recognition: a survey. Syst. Man Cybern. C Appl. Rev. IEEE Trans. 37(3), 311–324 (2007)

    Article  Google Scholar 

  16. Wang, C.-C., Wang, K.-C.: Hand posture recognition using adaboost with SIFT for human robot interaction. In: Recent Progress in Robotics: Viable Robotic Service to Human, pp. 317–329. Springer (2008)

  17. Kulkarni, V.S., Lokhande, S.: Appearance based recognition of american sign language using gesture segmentation. Int. J. Comput. Sci. Eng. 2(3), 560–565 (2010)

    Google Scholar 

  18. Caridakis, G., Karpouzis, K., Drosopoulos, A., Kollias, S.: SOMM: self organizing Markov map for gesture recognition. Pattern Recog. Lett. 31(1), 52–59 (2010)

    Article  Google Scholar 

  19. Stergiopoulou, E., Papamarkos, N.: Hand gesture recognition using a neural network shape fitting technique. Eng. Appl. Artif. Intell. 22(8), 1141–1158 (2009)

    Article  Google Scholar 

  20. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE (2001)

  21. Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: IEEE (2002)

  22. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media (2008)

  23. Zhang, C., Zhang, Z.: A survey of recent advances in face detection. Microsoft Research (2010)

  24. Yang, J., Lu, W., Waibel, A.: Skin-color modeling and adaptation. In: Computer Vision—ACCV’98, pp. 687–694 (1997)

  25. Chai, D., Ngan, K.: Face segmentation using skin-color map in videophone applications. IEEE Trans. Circ. Syst. Video Technol. 9, 551–564 (1999)

    Article  Google Scholar 

  26. Suzuki, S., and others: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985)

    Article  MATH  Google Scholar 

  27. Freeman, E., Brewster, S.: Messy tabletops: clearing up the occlusion problem. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems (2013)

  28. Jain, A.: Fundamentals of Digital Image Processing. Prentice-Hall, Inc. (1989)

  29. Ritter, G., Wilson, J.: Handbook of Computer Vision Algorithms in Image Algebra. CRC (2001)

  30. Atsalakis, A., Papamarkos, N.: Color reduction and estimation of the number of dominant colors by using a self-growing and self-organized neural gas. Eng. Appl. Artif. Intell. 19(7), 769–786 (2006)

    Article  Google Scholar 

  31. Kohonen, T.: The self-organizing map. Proc. IEEE 78(9), 1464–1480 (1990)

    Article  Google Scholar 

  32. Fritzke, B., others: A growing neural gas network learns topologies. Adv. Neural Inf. Process. Syst. 7, 625–632 (1995)

    Google Scholar 

  33. Fritzke, B.: Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1994)

    Article  Google Scholar 

  34. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257–286 (1989)

    Article  Google Scholar 

  35. Liu, N., Lovell, B.C., Kootsookos, P.J., Davis, R.I.: Model structure selection & training algorithms for an HMM gesture recognition system. In: IEEE (2004)

  36. Wilson, A.D., Bobick, A.F.: Parametric hidden markov models for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 21, 884–900 (1999)

    Article  Google Scholar 

  37. Abdul, Y.F., Wong, F.: Hidden Markov Model-based gesture recognition with overlapping hand-head/hand-hand estimated using kalman filter. In: IEEE (2012)

  38. Kohavi, R., others: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International Joint Conference on Artificial Intelligence (1995)

Download references

Author information

Authors and Affiliations

  1. Department of Electrical & Computer Engineering, Electric Circuits Analysis Laboratory, Democritus University of Thrace, 67100, Xanthi, Greece

    Kyriakos Sgouropoulos, Ekaterini Stergiopoulou & Nikos Papamarkos

Authors
  1. Kyriakos Sgouropoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ekaterini Stergiopoulou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nikos Papamarkos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikos Papamarkos.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sgouropoulos, K., Stergiopoulou, E. & Papamarkos, N. A Dynamic Gesture and Posture Recognition System. J Intell Robot Syst 76, 283–296 (2014). https://doi.org/10.1007/s10846-013-9983-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-013-9983-7

Keywords

Search

Navigation