Skip to main content

Advertisement

SpringerLink
Log in

Intelligent visual mouse system based on hand pose trajectory recognition in video sequences

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Hand gesture recognition based on computer vision is considered as an efficient approach to establish communication between human and computer. This research provides a novel system to recognize dynamic hand gestures as different functions of mouse. For this purpose, a white glove is utilized in which fingertips have five different colors. Then, based on the functions of the mouse, 11 dynamic hand gestures are defined. In order to track the hand in each frame, the optical flow and GMM algorithms are used. Then, by using the mean and variance of the colors in each RGB plane, trajectories of the fingertips are detected. To make the acquired information richer, the Representative Trajectory of these five trajectories is computed. Features are extracted from the curves through a process inspired by the concept of shape context. In this process, each trajectory is normalized and then the histogram of extracted vector from normalized curve is calculated in a log-polar space. Thus, by this descriptor, a feature vector with length of 672 is created. Using the defined 11 dynamic gestures, a dataset including 220 observations is constructed and by using the aforementioned features, the relevant training matrix is formed. By employing PCA and Sequential Feature Selection techniques, dimension of the feature vector is reduced. For the experiment, different classifiers are applied and the experimental results confirm the privileged performance of the proposed system for the intelligent visual mouse.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

References

  1. Lee, J.Y., Rhee, G.W., Seo, D.W.: Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment. Int. J. Adv. Manuf. Technol. 51, 1069–1082 (2010)

    Article  Google Scholar 

  2. Hu, C., Liang, L., Ma, S., Lu, H.: Virtual Mouse—Inputting Device by Hand Gesture Tracking and Recognition. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing (2000)

  3. Rautaray, S.S., Agrawal, A.: Interaction with virtual game through hand gesture recognition. In: International conference on multimedia, signal processing and communication technologies, 2011

  4. Starner, T., Auxier, J., Ashbrook, D., Gandy, M.: The gesture pendant: a self-illuminating, wearable, infrared computer vision system for home automation control and medical monitoring. In: Proc. of IEEE int. symposium on 2000

  5. Agris, U.V., Zieren, J., Canzler, U., Bauer, B., Kraiss, K.-F.: Recent developments in visual sign language recognition. Univ. Access Inf. Soc. 6, 323–362 (2008)

    Article  Google Scholar 

  6. Kim, J.-H., Thang, N. D., Kim, E.-S.: 3-D hand motion tracking and gesture recognition using a data glove. In: IEEE international symposium on industrial electronics. Seoul (2009)

  7. Murthy, G.R.S., Jadon, R.S.: A review of vision based hand gestures recognition. Int. J. Inf. Technol. Knowl. Manag. 2(2), 405–410 (2009)

    Google Scholar 

  8. Zhao, Z.-Y., Gao, W.-L., Zhu, M.-M., Yu, L.: A vision based method to distinguish and recognize static and dynamic gesture. In: International workshop on information and electronics engineering (IWIEE) (2012)

  9. Ramamoorthy, A., Vaswani, N., Chaudhury, S., Banerjee, S.: Recognition of dynamic hand gestures. Pattern Recogn. 36, 2069–2081 (2003)

    Article  MATH  Google Scholar 

  10. Maccormick, J., Blake, A.: A probabilistic contour discriminant for object localisation, pp. 390–395 (1998)

  11. Alon, J., Athitsos, V., Yuan, Q., Sclaroff, S.: Simultaneous localization and recognition of dynamic hand gestures. In: Proc. IEEE workshop on motion and video computing (2005)

  12. Müller, M.: Dynamic time warping. Information retrieval for music and motion, pp. 69–82. Springer, Berlin (2007)

    Chapter  Google Scholar 

  13. Bao, J., Song, A., Guo, Y., Tang, H.: Dynamic hand gesture recognition based on SURF tracking. In: Electric information and control engineering (ICEICE), Wuhan (2011)

  14. Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: SURF: speeded up robust features. Comput. Vision Image. Underst. (CVIU) 110(3), 346–359 (2008)

    Article  Google Scholar 

  15. Laptev, I., Indeberg, T.L.: Space-time interest points. In: The ninth ieee international conference on computer vision (ICCV) (2003)

  16. Shen, X., Hua, G., Williams, L., Wu, Y.: Dynamic hand gesture recognition: an exemplar-based approach from motion divergence fields. Image Vis. Comput. 30, 227–235 (2011)

    Article  Google Scholar 

  17. Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proc. of British machine vision conference (2002)

  18. Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: Computer vision and pattern recognition (2006)

  19. Lienhart, R., Maydt, J.: An extended set of haar-like features for rapid object detection. In: IEEE ICIP (2002)

  20. Rojas, R.: AdaBoost and the super bowl of classi a tutorial introduction to adaptive boosting. Computer Science Departmen, Freie Universitat, Berlin (2009)

    Google Scholar 

  21. Stern, H., Shmueli, M., Berman, S.: Most discriminating segment—longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification. Pattern Recognit. Lett. 34(15), 1980–1989 (2013)

  22. Li, H., Greenspan, M.: Model-based segmentation and recognition of dynamic gestures in continuous video streams. Pattern Recogn. 44, 1614–1628 (2011)

    Article  Google Scholar 

  23. Bhuyan, M., Ghosh, D., Bora, P.: Feature extraction from 2D gesture trajectoy in dynamic hand gesture recognition. Cybernetics and intelligent systems. Chulalongkorn Univ, Bangkok (2006)

    Google Scholar 

  24. Bhuyan, M.K., Ghosh, D., Bora, P.K.: Estimation of 2D motion trajectories from video object planes and its application in hand gesture recognition. Pattern recognition and machine intelligence. Indian Statistical Institute, Kolkata (2005)

    Google Scholar 

  25. Barron, J.L., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. IEEE 1, 236–242 (1992)

    Google Scholar 

  26. Fleet, D.J., Weiss, Y.: Optical flow estimation, handbook of mathematical models in computer vision. Springer, New York (2006)

    Google Scholar 

  27. Belongie, S., Malik, J.: Matching with shape contexts. In: IEEE workshop on content based access of image and video libraries (2000)

  28. de Boor, C.: A practical guide to splines. Springer-Verlag, New York (1978)

    Book  MATH  Google Scholar 

  29. Kohavi, R.: A study of cross validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the fourteenth international joint conference on artificial intelligence, San Mateo (1995)

  30. de La Gorce, M., Paragios, N.: A variational approach to monocular hand-pose estimation. Comput. Vis. Image Underst. 114, 363–372 (2010)

    Article  Google Scholar 

  31. Elmezain, M., Al-Hamadi, A., Michaelis, B.: A robust method for hand gesture segmentation and recognition using forward spotting scheme in conditional random fields. In: international conference on pattern recognition, Istanbul (2010)

  32. Wenjun, T., Chengdong, W., Shuying, Z., Li, J.: Dynamic hand gesture recognition using motion trajectories and key frames. In: 2nd international conference on advanced computer control (ICACC), Shenyang (2010)

  33. Kao, C.-Y., Fahn, C.-S.: A human-machine interaction technique: hand gesture recognition based on hidden markov models with trajectory of hand motion. Procedia Eng 15, 3739–3743 (2011)

    Article  Google Scholar 

  34. Al-Hamadi, A., Elmezain, M., Michaelis, B.: Hand gesture recognition based on combined features extraction. Int. J. Inf. Math. Sci. 6(1), 1–6 (2010)

    Google Scholar 

  35. Lamberti, L., Camastra, F.: Handy: A real-time three color glove-based gesture recognizer with learning vector quantization. Expert Syst. Appl. 39(12), 10489–10494 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision Res. Lab, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran

    Behnam Maleki & Hossein Ebrahimnezhad

Authors
  1. Behnam Maleki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Ebrahimnezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Ebrahimnezhad.

Additional information

Communicated by B. Prabhakaran.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maleki, B., Ebrahimnezhad, H. Intelligent visual mouse system based on hand pose trajectory recognition in video sequences. Multimedia Systems 21, 581–601 (2015). https://doi.org/10.1007/s00530-014-0420-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0420-y

Keywords

Search

Navigation