Skip to main content
SpringerLink
Log in

A virtual mouse interface with a two-layered Bayesian network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

During the last decade, many natural interaction methods between human and computer have been introduced. They were developed for substitutions of keyboard and mouse devices so that they provide convenient interfaces. Recently, many studies on vision based gestural control methods for Human-Computer Interaction (HCI) have been attracted attention because of their convenience and simpleness. Two of the key issues in these kinds of interfaces are robustness and real-time processing. This paper presents a hand gesture based virtual mouse interface and Two-layer Bayesian Network (TBN) for robust hand gesture recognition in real-time. The TBN provides an efficient framework to infer hand postures and gestures not only from information at the current time frame, but also from the preceding and following information, so that it compensates for erroneous postures and its locations under cluttered background environment. Experiments demonstrated that the proposed model recognized hand gestures with a recognition rate of 93.76 % and 85.15 % on simple and cluttered background video data, respectively, and outperformed previous methods: Hidden Markov Model (HMM), Finite State Machine (FSM).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Argyros A, Lourakis M (2006) Vision-based interpretation of hand gestures for remote control of a computer mouse. In: Proc European conference on computer vision, vol 3979. Graz, Austria, pp 40–51

  2. Brand J, Mason J (2000) A comparative assessment of three approaches to pixel level human skin-detection 1:1056–1059

  3. Bretzner L, Laptev I, Lindeberg T, Lenman S, Sundblad Y (2001) A prototype system for computer vision based human computer interaction, Tech. Rep. CVAP251, Department of Numerical Analysis and Computer Science, KTH (Royal Institute of Technology), Stockholm, Sweden

  4. Caschera M C, Ferri F, Grifoni P (2013) InteSe: an integrated model for resolving ambiguities in multimodal sentences. IEEE Trans Syst Man Cybern Syst 43 (4):911–931

    Article  Google Scholar 

  5. Dempster A, Laird N, Rubin D (1977) Maximum-likelihood from incomplete data via the EM algorithm. J R Stat Soc 39(1):1–38

    MathSciNet  MATH  Google Scholar 

  6. Dorner B, Hagen E (1994) Towards an American sign language interface. Artif Intell Rev 8(2):235–253

    Article  Google Scholar 

  7. El-Sawah A, Georganas N, Petriu E (2008) A prototype for 3-D hand tracking and posture estimation. IEEE Trans Instrument Measur 57(8):1627–1636

    Article  Google Scholar 

  8. Fu Y, Huang T S (2007) hMouse: Head tracking driven virtual computer mouse. In: Proc. of the 8th IEEE workshop on applications of computer vision. Austin, pp 30–36

  9. Hu K, Yin L (2015) Multiple feature representations from multi-layer geometric shape for hand gesture analysis. In: Proc. of 11th IEEE international conference on automatic face and gesture recognition. Slovenia

  10. Isard M, Blake A (1998) Condensation - conditional density propagation for visual tracking. Int J Comput Vis 29:5–28

    Article  Google Scholar 

  11. Isard M, Maccormick J (2001) BraMBLe: a Bayesian multiple-blob tracker 2:34–41

  12. Krejov P, Gilbert A, Bowden R (2015) Combining discriminative and model based approaches for hand pose estimation. In: Proc. of 11th IEEE international conference on automatic face and gesture recognition. Slovenia

  13. Laptev I, Lindeberg T (2001) Tracking of multi-state hand models using particle filtering and a hierarchy of multi-scale image features, pp 63–74

  14. Marcel S (1996) http://www.idiap.ch/resources/gestures/

  15. Mitra S, Acharya T (2007) Gesture recognition: a survey. IEEE Trans Syst Man Cybern 37(3):311–324

    Article  Google Scholar 

  16. Murphy K (2002) Dynamic Bayesian networks: representation, inference and learning, Ph.D. thesis, University of British Columbia

  17. Nguyen V-T, Le T-L, Tran T-H, Mullot R, Courboulay V (2015) A new hand representation based on kernels for hand posture recognition. In: Proc. of 11th IEEE international conference on automatic face and gesture recognition. Slovenia

  18. Raheja JL, Shyam R, Kumar U, Prasad P B (2010) Real-time robotic hand control using hand gestures. Int Conf Mach Learn Comput:12–16

  19. Ramamoorthy A, Vaswani N, Chaudhury S, Banerjee S (2003) Recognition of dynamic hand gestures. Pattern Recog 36(9):2069–2081

    Article  MATH  Google Scholar 

  20. Robertson P, Laddaga R, Kleek M (2004) Virtual mouse vision based interface, pp 177–193

  21. Stenger B, Thayananthan A, Torr P, Cipolla R (2006) Model-based hand tracking using a hierarchical Bayesian filter. IEEE Trans Pattern Anal Mach Intell 28 (9):1372–1384

    Article  MATH  Google Scholar 

  22. Suk H-I, Shin B-K, Lee S-W (2010) Hand gesture recognition based on dynamic Bayesian network framework 43(9):3059–3072

  23. Vafadar M, Behrad A (2008) Human hand gesture recognition using motion orientation histogram for interaction of handicapped persons with computer. In: Proc. of international conference on image and signal processing, vol 5099. Normandy, France, pp 378–385

  24. Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  25. Yeasin M, Chaudhuri S (2000) Visual understanding of dynamic hand gestures. Pattern Recog 33(11):1805–1817

    Article  Google Scholar 

Download references

Acknowledgments

This work was partly supported by the ICT R&D program of MSIP/IITP [B0101-15-0552 , Development of Predictive Visual Intelligence Technology] and also supported by the Implementation of Technologies for Identification, Behavior, and Location of Human based on Sensor Network Fusion Program through the Ministry of Trade, Industry and Energy (Grant No. 10041629).

Author information

Authors and Affiliations

  1. S-1, S-1 Building, 25, Sejong-daero 7-gil, Jung-gu, Seoul, 100-733, South Korea

    Myung-Cheol Roh

  2. Department of Brain and Cognitive Engineering, Korea University, Anam-dong, Seongbuk-ku, Seoul, 136-713, South Korea

    Dongoh Kang, Sungju Huh & Seong-Whan Lee

Authors
  1. Myung-Cheol Roh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongoh Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sungju Huh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seong-Whan Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seong-Whan Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Roh, MC., Kang, D., Huh, S. et al. A virtual mouse interface with a two-layered Bayesian network. Multimed Tools Appl 76, 1615–1638 (2017). https://doi.org/10.1007/s11042-015-3144-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3144-x

Keywords

Search

Navigation