Skip to main content

Advertisement

SpringerLink
Log in

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aaron F, Andrew D (1997) A state-based approach to the representation and recognition of gesture. IEEE Trans Pattern Anal and Mach Intell 19(12): 1325–1337

    Article  Google Scholar 

  • Aas K, Eikvil L, Huseby RB et al (1999) Application of hidden Markov chains in image analysis. Pattern Recognit 32(4): 703

    Article  Google Scholar 

  • AL-Rousan M, Assaleh K, AL-Rousan T (2009) Video-based signer-independent Arabic sign language recognition using hidden Markov models. Appl Soft Comput 9(3): 990–999

    Article  Google Scholar 

  • Aran O, Ari I, Akarun L, Sankur B, Benoit A, Caplier A, Campr P, Carrillo A, Fanarda F (2009) Sign tutor: an interactive system for sign language tutoring. IEEE Multimed 1: 81–93

    Article  Google Scholar 

  • Assaleh K, Shanableh T, Fanaswala M, Amin F, Bajaj H (2010) Continuous Arabic sign language recognition in user dependent mode. J Intell Learn Syst Appl 2: 19–27

    Google Scholar 

  • Bauer B, Kraiss K-F (2001) Towards an automatic sign language recognition system using subunits. Gesture workshop, pp 64–75

  • Bilal S, Akmelawati R, Salami MJE, Shafie AA, Bouhabba EM (2010) A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking. In: Proceedings of international conference on mechatronics and automation (ICMA), Xi’an, August 2010, pp 934–939

  • Black MJ, Jepson AD (1998) A probabilistic framework for matching temporal trajectories: condensation-based recognition of gesture and expressions. In: Proceedings fifth European conference computer vision, pp 909–924

  • Braffort A (1996) ARGO: an architecture for sign language recognition and interpretation. In: Harling P, Edwards A (eds) Progress in gestural interaction. Springer, Berlin, pp 17–30

    Google Scholar 

  • Brashear H, Park K-H, Lee S, Henderson V, Hamilton H, Starner T (2006) American sign language recognition in game development for deaf children. In: Proceedings of the Assets, pp 79–86

  • Del Rose MS, Wagner CC (2011) Survey on classifying human actions through visual sensors. J Artif Intell Rev. doi:10.1007/s10462-011-9232-z

  • Derpanis KG (2004) A review of vision-based hand gestures. Internal report, Department of Computer Science, York University

  • Elmezain M, Al-Hamadi A, Appenrodt J, Michaelis B (2008a) A hidden Markov model-based continuous gesture recognition system for hand motion trajectory. In: 19th international conference on pattern recognition, ICPR 2008, pp 1–4

  • Elmezain M, Al-Hamadi A, Michaelis B (2008b) Real-time capable system for hand gesture recognition using hidden markov models in stereo color image sequences. An international journal of algorithms, data structures and techniques for computer graphics and visualization, modeling, CAD & GIS systems, computer vision, image processing and pattern recognition, human computer interaction, animation and virtual reality, multimedia systems and applications in parallel, distributed and mobile environment (JWSCG) 2008 16(1): 65–72

    Google Scholar 

  • Elmezain M, Al-Hamadi A, Appenrodt J, Michaelis B (2009) A hidden Markov model-based isolated and meaningful hand gesture recognition. Int J Electr Comput Syst Eng 3:3

    Google Scholar 

  • Fang GL, Gao W (2002) A SRN/HMM system for signer independent continuous sign language recognition. In: Proceedings of the fifth international conference on automatic face and gesture recognition, pp 312–317

  • Fang GL, Gao XJ, Gao W, Chen YQ (2004) A novel approach to automatically extracting basic units from Chinese sign language. In: 17th international conference on pattern recognition (ICPR), Cambridge, England, pp 454–457

  • Fels SS, Hinton GE (1993) Glove-talk: a neural network interface between a data-glove and a speech synthesizer. IEEE Trans Neural Netw 4(1): 2–8

    Article  Google Scholar 

  • Fels SS, Hinton GE (1997) \({^{\rm \underline{a}}}\) Glove-talk II: “a neural network interface which maps gestures to parallel format speech synthesizer controls”. IEEE Trans Neural Netw 9(1): 205–212

    Article  Google Scholar 

  • Garg P, Aggarwal N, Sofat S (2009) Vision based hand gesture recognition. World Acad Sci Eng Technol 49: 973–977

    Google Scholar 

  • Gao W, Ma JY, Wu JQ, Wang CL (2000) Sign language recognition based on HMM/ANN/DP. Int J Pattern Recognit Artif Intell 14(5): 587–602

    Article  Google Scholar 

  • Gao W, Fang G, Zhao D, Chen Y (2004) Transition movement models for large vocabulary continuous sign language recognition. In: Proceedings of the sixth IEEE international conference automatic face and gesture recognition, pp 553–558

  • Gao W, Fanga G, Zhaoa D, Chen Y (2004) A Chinese sign language recognition system based on SOFM/SRN/HMM. Pattern Recognit 37:2389–2402

    Google Scholar 

  • Grobel K, Assan M (1997) Isolated sign language recognition using hidden Markov models. In: IEEE international conference on systems, man, and cybernetics, computational cybernetics and simulation, vol 1. pp 162–167

  • Gruenstein A (2002) Two methods of gesture recognition. Retrieved from http://www.mit.edu/~alexg/vision/review.pdf

  • Hidden Markov Model Toolkit (HTK)—Speech Recognition toolkit, (2011). Retrieved from June, 2011. http://htk.eng.cam.ac.uk/

  • Holden EJ, Lee G, Owens R (2005) Australian sign language recognition. Mach Vis Appl 16: 312–320

    Article  Google Scholar 

  • Hunter E, Schlenzig J, Jain R (1995) Posture estimation in reduced-model gesture input systems. In: Proceedings of the international workshop on automatic face-and gesture-recognition, pp 290–295

  • Isard M, Blake A (1998) Condensation—conditional density propagation for visual tracking. Int J Comput Vis 29(1): 5–28

    Article  Google Scholar 

  • Just A, Bernier O, Marcel S (2004) HMM and IOHMM for the recognition of mono- and bi-manual 3D hand gestures. In: British machine vision conference (BMVC)

  • Keskin C, Erkan A, Akarun L (2003) Real time hand tracking and 3D gesture recognition for interactive interfaces using HMM. In: Proceedings of international conference on artificial neural networks

  • Kim J-B, Park K-H, Bang W-C, Bien ZZ (2002a) Continuous korean sign language recognition using gesture segmentation and hidden markov modeL. In: RESNA 25th international conference on technology disability minneapolis USA

  • Kim J-B, Park K-H, Bang W-C, Bien ZZ (2002b) Continuous gesture recognition system for Korean sign language based on fuzzy logic and hidden Markov model. In: Proceedings of the 2002 IEEE international conference on fuzzy systems, FUZZ-IEEE’02, USA, pp. 1574–1579

  • Kobayashi T, Haruyama S (1997) Partly-hidden Markov model and its application to gesture recognition. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing (ICASSP) vol 4, pp 3081–3084

  • Lee H-K, Kim JH (1999) An HMM-based threshold model approach for gesture recognition. IEEE Trans Pattern Anal Mach Intell 21(10): 961–973

    Article  Google Scholar 

  • Liang R-H, Ouhyoung M (1996) A sign language recognition system using hidden Markov model and context sensitive search. In: Proceedings of the ACM symposium on virtual reality software and technology. ACM Press, pp 59–66

  • Liang R-H, Ouhyoung M (1998) A real-time continuous gesture recognition system for sign language. In: Proceedings of the third IEEE international conference on automatic face and gesture recognition, 1998, pp 558–567

  • Liddell SK, Johnson RE (1989) American sign language: the phonological base. Sign Lang Stud 64: 195–277

    Article  Google Scholar 

  • Ma J, Gao W, Wu J, Wang C (2000) A continuous Chinese sign language recognition system, In: Proceedings fourth IEEE international conference on automatic face and gesture recognition, 2000, pp 428–433

  • Maebatake M, Suzuki I, Nishida M, Horiuchi Y, Kuroiwa S (2008) Sign language recognition based on position and movement using multi-stream HMM. In: Second international symposium on universal communication

  • Manresa C, Varona J, Mas R, Perales FJ (2000) Real-time hand tracking and gesture recognition for human-computer interaction. Electron Lett Comput Vis Image Anal 0(0): 1–7

    Google Scholar 

  • Mitra S, Acharya T (2007) Gesture recognition: a survey, systems, man, and cybernetics, part C: applications and reviews. IEEE Trans 37: 311–324

    Google Scholar 

  • Munib Q, Habeeb M, Takruri B, Al-Malik HA (2007) American sign language (ASL) recognition based on hough transform and neural networks. Expert Syst Appl 32: 24–37

    Article  Google Scholar 

  • Murthy GRS, Jadon RS (2009) A review on vision based Hand gestures recognition. Int J Inf Technol Knowl Manag 2: 405–410

    Google Scholar 

  • Nam Y, Wohn KY (1996) Recognition of space-time hand-gestures using hidden Markov model. In: ACM symposium on virtual reality software and technology, pp 51–58

  • Nam Y, Wohn KY (1999) Recognition and modeling of hand gestures using colored petri nets. IEEE Trans Syst Man Cybern 29: 514–521

    Article  Google Scholar 

  • Nguyen DB, Enokida S, Toshiaki E (2005) Real-time hand tracking and gesture recognition system. IGVIP05 conference, CICC, pp 362–368

  • Nianjun L, Brian CL, Peter JK, Richard AD (2004) Model structure selection & training algorithms for a HMM gesture recognition system. In: International IWFHR, pp 100–106

  • Philippine Federation of the Deaf (2005) Filipino sign language: a compilation of signs from regions of the Philippines, Part 1. LFS Printing Services, Inc., Quezon City

  • Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2): 257–285

    Article  Google Scholar 

  • Ramamoorthya A, Vaswania N, Chaudhurya S, Banerjee S (2003) Recognition of dynamic hand gestures. Pattern Recognit 36: 2069–2081

    Article  Google Scholar 

  • Rigoll G, Kosmala A (1997) New improved feature extraction methods for real-time high performance image sequence recognition. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing (ICASSP), Munich, pp 2901–2904

  • Rigoll G, Kosmala A, Eickeler S (1997) High performance real-time gesture recognition using hidden Markov models. In: Proceedings of the Gesture workshop, Bielefeld, Germany

  • Rigoll G, Kosmala A, Eickeler S (1998) Hidden Markov model based continuous online gesture recognition. In: Internation conference on pattern recognition (ICPR), vol 2. pp 1206–1208

  • Sandjaja IN, Marcos N (2009) Sign language number recognitio. INC, IMS and IDC, NCM. In: Fifth international joint conference, pp 1503–1508

  • Starner T, Pentland A (1995a) Visual recognition of American sign language using hidden Markov models. Technical report TR-306, Media Lab, MIT

  • Starner T, Pentland A (1995b) Real-time American sign language recognition from video using hidden Markov models. Technical report TR-306, Media Lab, MIT

  • Tanibata N, Shimada N (2002) Extraction of hand features for recognition of sign language words. In: International conference on vision interface, pp 391–398

  • The Georgia Tech Gesture Toolkit (GT2k) (2011). Retrieved from June, 2011. http://gt2k.cc.gatech.edu/

  • Vassilia NP, Konstantinos GM (2003) On feature extraction and sign recognition for greek sign language. In: Proceedings of the 7th IASTED international conference artificial intelligence and soft computer, pp 93–98

  • Vogler C, Metaxas D (1997) Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods. In: Proceedings of the IEEE international conference on systems, man and cybernetics, pp 156–161

  • Vogler C, Metaxas D (1997) Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods. In: Proceedings of the IEEE international conference on systems, man and cybernetics, Orlando, FL, pp 156–161

  • Vogler C, Metaxas D (1998) ASL recognition based on a coupling between HMMs and 3D motion analysis. In: Proceedings of the international conference on computer vision. Mumbai, India, pp 363–369, January 4–7

  • Vogler C, Metaxas D (1999a) Toward scalability in ASL recognition: breaking down signs into phonemes. In: Gesture-based communication in human-computer interaction vol 1739, Lecture notes in artificial intelligence. Springer, Berlin, pp 211–224

  • Vogler C, Metaxas D (1999b) Parallel hidden Markov models for American sign language Recognition. In: Proceedings of the IEEE international conference on computer vision, Kerkyra, Greece, pp 116–122

  • Vogler C, Metaxas D (2000) A framework for recognizing the simultaneous aspects of American sign language. J Comput Vis Image Underst 81(3): 358–384

    Article  Google Scholar 

  • Wu Y, Huang TS (2001) Hand modeling analysis and recognition for vision-based human computer interaction. IEEE Signal Process Mag Special Issue Immers Interact Tech 18(3): 51–60

    Google Scholar 

  • Yamato J, Ohya J, Ishii K (1992) Recognizing human action in time-sequential images using hidden Markov models. In: Proceedings of computer vision and pattern recognition (CVPR), Champaign, IL, pp 379–385

  • Yamato J, Ohya J, Ishii K (1992) Recognizing human action in time sequential images using hidden Markov model. In: Proceedings of the IEEE international conference computer vision and pattern recognition, Champaign, IL, pp 379–385

  • Yang M-H, Ahuja N (1999) Recognizing hand gesture using motion trajectories. In: IEEE CS conference on computer vision and pattern recognition, vol 1. pp 466–472

  • Yoon H-S, Soh J, Bae YJ, Yang HS (2001) Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognit 34:1491–1501

    Google Scholar 

  • Zafrulla Z, Brashear H, Yin P, Presti P, Starner T, Hamilton H (2010) American sign language phrase verification in an educational game for deaf children. In: International conference on pattern recognition, pp 3846–3849

  • Zhang L-G, Chen Y, Fang G, Chen X, GaoW(2005a) A vision-based sign language recognition system using tied-mixture density HMM. In: Proceedings of the 6th international conference on multimodal interfaces (ICMI)

  • Zhang L-G, Chen X, Wang C, Chen Y, Gao W (2005b) Recognition of sign language subwords based on boosted hidden Markov models. In: Proceedings of the 6th international conference on multimodal interfaces (ICMI)

Download references

Author information

Authors and Affiliations

  1. Department of Mechatronics Engineering, International Islamic University Malaysia (IIUM), Jl Gombak, 53100, Kuala Lumpur, Malaysia

    Sara Bilal, Rini Akmeliawati, Amir A. Shafie & Momoh Jimoh E. Salami

Authors
  1. Sara Bilal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rini Akmeliawati
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Amir A. Shafie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Momoh Jimoh E. Salami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sara Bilal.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bilal, S., Akmeliawati, R., Shafie, A.A. et al. Hidden Markov model for human to computer interaction: a study on human hand gesture recognition. Artif Intell Rev 40, 495–516 (2013). https://doi.org/10.1007/s10462-011-9292-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-011-9292-0

Keywords

Search

Navigation