Abstract
This paper presents a real time dynamic hand gesture and posture recognition system based on a neural network and a Hidden Markov Model. For skin color segmentation an adaptive online trained skin color model is used, while the hand posture recognition is accomplished through a likelihood-based classification technique of geometric features. A novel trajectory smoothing technique based on Self Organized Neural Network is introduced to improve HMM classification performance of dynamic gestures. The aim of the proposed system is the creation of a visual dictionary combining hand postures and dynamic gestures. The system has been successfully tested with many people under varying light conditions and different web cameras.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Murthy, G., Jadon, R.: A review of vision based hand gestures recognition. Int. J. Inf. Technol. Knowl. Manag. 2(2), 405–410 (2009)
Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.-F., Kirbas, C., McCullough, K.E., Ansari, R.: Multimodal human discourse: gesture and speech. ACM Trans. Comput-Hum. Interact. (TOCHI). 9(3), 171–193 (2002)
Van den Bergh, M., Van Gool, L.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. In: 2011 IEEE Workshop on Applications of Computer Vision (WACV) (2011)
Elmezain, M., Al-Hamadi, A., Appenrodt, J., Michaelis, B.: A hidden markov model-based continuous gesture recognition system for hand motion trajectory. In: 19th International Conference on Pattern Recognition, ICPR (2008)
Kollorz, E., Penne, J., Hornegger, J., Barke, A.: Gesture recognition with a time-of-flight camera. Int. J. Intell. Syst. Technol. Appl. 5(3), 334–343 (2008)
Kakumanu, P., Makrogiannis, S., Bourbakis, N.: A survey of skin-color modeling and detection methods. Pattern Recognit. 40(3), 1106–1122 (2007)
Vezhnevets, V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. Graphicon (2003)
Doulamis, N., Doulamis, A., Kosmopoulos, D.: Content-based decomposition of gesture videos. In: IEEE Workshop on Signal Processing Systems Design and Implementation (2005)
Araki, R., Gohshi, S., Ikenaga, T.: Real-time both hands tracking using CAMshift with motion mask and probability reduction by motion prediction. In: Signal & Information Processing Association Annual Summit and Conference, (APSIPA ASC) 2012. Asia-Pacific (2012)
Oikonomidis, I., Kyriazis, N., Argyros, A.: Efficient model-based 3d tracking of hand articulations using kinect. In: British Machine Vision Conference (2011)
Athitsos, V., Wang, H., Stefan, A.: A database-based framework for gesture recognition. Pers. Ubiquit. Comput. 14(6), 511–526 (2010)
Malassiotis, S., Strintzis, M.: Real-time hand posture recognition using range data. Image Vis. Comput. 26(7), 1027–1037 (2008)
Erol, A., Bebis, G., Nicolescu, M., Boyle, R.D., Twombly, X.: Vision-based hand pose estimation: a review. Comp. Vision Image Underst. 108(1), 52–73 (2007)
Hasan, M.M., Mishra, P.K.: Hand gesture modeling and recognition using geometric features: a review. Can. J. Image Process. Comput. Vis. 3(1), 12–26 (2012)
Mitra, S., Acharya, T.: Gesture recognition: a survey. Syst. Man Cybern. C Appl. Rev. IEEE Trans. 37(3), 311–324 (2007)
Wang, C.-C., Wang, K.-C.: Hand posture recognition using adaboost with SIFT for human robot interaction. In: Recent Progress in Robotics: Viable Robotic Service to Human, pp. 317–329. Springer (2008)
Kulkarni, V.S., Lokhande, S.: Appearance based recognition of american sign language using gesture segmentation. Int. J. Comput. Sci. Eng. 2(3), 560–565 (2010)
Caridakis, G., Karpouzis, K., Drosopoulos, A., Kollias, S.: SOMM: self organizing Markov map for gesture recognition. Pattern Recog. Lett. 31(1), 52–59 (2010)
Stergiopoulou, E., Papamarkos, N.: Hand gesture recognition using a neural network shape fitting technique. Eng. Appl. Artif. Intell. 22(8), 1141–1158 (2009)
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE (2001)
Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: IEEE (2002)
Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media (2008)
Zhang, C., Zhang, Z.: A survey of recent advances in face detection. Microsoft Research (2010)
Yang, J., Lu, W., Waibel, A.: Skin-color modeling and adaptation. In: Computer Vision—ACCV’98, pp. 687–694 (1997)
Chai, D., Ngan, K.: Face segmentation using skin-color map in videophone applications. IEEE Trans. Circ. Syst. Video Technol. 9, 551–564 (1999)
Suzuki, S., and others: Topological structural analysis of digitized binary images by border following. Comput. Vis. Graph. Image Process. 30(1), 32–46 (1985)
Freeman, E., Brewster, S.: Messy tabletops: clearing up the occlusion problem. In: CHI’13 Extended Abstracts on Human Factors in Computing Systems (2013)
Jain, A.: Fundamentals of Digital Image Processing. Prentice-Hall, Inc. (1989)
Ritter, G., Wilson, J.: Handbook of Computer Vision Algorithms in Image Algebra. CRC (2001)
Atsalakis, A., Papamarkos, N.: Color reduction and estimation of the number of dominant colors by using a self-growing and self-organized neural gas. Eng. Appl. Artif. Intell. 19(7), 769–786 (2006)
Kohonen, T.: The self-organizing map. Proc. IEEE 78(9), 1464–1480 (1990)
Fritzke, B., others: A growing neural gas network learns topologies. Adv. Neural Inf. Process. Syst. 7, 625–632 (1995)
Fritzke, B.: Growing cell structures—a self-organizing network for unsupervised and supervised learning. Neural Netw. 7(9), 1441–1460 (1994)
Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 257–286 (1989)
Liu, N., Lovell, B.C., Kootsookos, P.J., Davis, R.I.: Model structure selection & training algorithms for an HMM gesture recognition system. In: IEEE (2004)
Wilson, A.D., Bobick, A.F.: Parametric hidden markov models for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 21, 884–900 (1999)
Abdul, Y.F., Wong, F.: Hidden Markov Model-based gesture recognition with overlapping hand-head/hand-hand estimated using kalman filter. In: IEEE (2012)
Kohavi, R., others: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: International Joint Conference on Artificial Intelligence (1995)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sgouropoulos, K., Stergiopoulou, E. & Papamarkos, N. A Dynamic Gesture and Posture Recognition System. J Intell Robot Syst 76, 283–296 (2014). https://doi.org/10.1007/s10846-013-9983-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10846-013-9983-7