Skip to main content

Advertisement

Springer Nature Link
Log in

Real-time dynamic gesture recognition and hand servo tracking using PTZ camera

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A technology of real-time dynamic gesture recognition and hand tracking using a Pan-Tilt-Zoom (PTZ) camera was presented in this study. It was aimed to achieve robust scheme that stably recognized simple hand gestures and tracked the hand by means of a PTZ camera to keep the fingertip remaining in the center of the camera. For this purpose, the hand region was initially segmented in a cluttered environment using skin color segmentation in YCbCr color space to get the silhouette of the hand. Furthermore, the Monte Carlo Sampling method was used to estimate the Cubic Bezier curves best fitted to the sub contour points centralized in each contour point, and the fingertips were detected by combining the local maximums of a cumulative curvature with detection of convex defects. After that, feature triangle analysis was utilized to achieve dynamic recognition of simple gestures including “right click down” and “right click up”. Finally, the PTZ camera was driven by the algorithm to achieve servo tracking with the target fingertip when the gesture “right click down” was detected. As is shown by the experimental results, the proposed approach recognized the dynamic gestures and located the fingertips’ positions precisely, and realized the follow-up servo tracking using PTZ camera in real-time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

References

  1. Anderson KR (1978) A reevaluation of an efficient algorithm for determining the convex hull of a finite planar set. Inf Process Lett 7(1):53–55

    Article  MathSciNet  MATH  Google Scholar 

  2. Argyros AA, Lourakis MIA (2006) Vision-based interpretation of hand gestures for remote control of a computer mouse. Lect Notes Comput Sci 3979:40–51

    Article  Google Scholar 

  3. Barrho J, Adam M, Kiencke U (2006) Finger localization and classification in images based on generalized hough transform and probabilistic models. Int Conf Control Autom Robot Vision IEEE 1–6

  4. Barros P et al (2017) A dynamic gesture recognition and prediction system using the convexity approach. Comput Vis Image Understanding 155:139–149

    Article  Google Scholar 

  5. Bhuyan MK, Neog DR, Kar MK (2011) Hand pose recognition using geometric features. Commun IEEE 1-5

  6. Chai D, Ngan KN (1998) Locating facial region of a head-and-shoulders color image. IEEE Int Conf Automatic Face Gesture Recognition 1998. Proc IEEE 124–129

  7. Erol A et al (2007) Vision-based hand pose estimation: a review. Comput Vis Image Underst 108(1):52–73

    Article  Google Scholar 

  8. Ge SS, Yang Y, Lee TH (2008) Hand gesture recognition and tracking based on distributed locally linear embedding. Image Vis Comput 26.12:1607–1620

    Article  Google Scholar 

  9. Gurav RM, Kadbe PK (2015) Real time finger tracking and contour detection for gesture recognition using Open CV. Proc IEEE Int Conf Ind Instrum Control 974–977

  10. Hartanto R, Kartikasari A (2017) Android based real-time static Indonesian sign language recognition system prototype. Int Conf Inf Technol Electr Eng IEEE 1–6

  11. Jo KH, Kuno Y, Shirai Y (1998) Manipulative hand gesture recognition using task knowledge for human computer interaction. Int Conf Face Gesture Recogn IEEE Comput Soc 468

  12. Kim J et al (2017) An adaptive local binary pattern for 3D hand tracking. Pattern Recogn 61:139–152

    Article  Google Scholar 

  13. Koike H, Sato Y, Kobayashi Y (2001) Integrating paper and digital information on enhanced desk: a method for realtime finger tracking on an augmented desk system. ACM Trans Comput Hum Interact (TOCHI) 8(4):307–322

    Article  Google Scholar 

  14. Letessier J (2004) Visual tracking of bare fingers for interactive surfaces. ACM Symp User Interface Software Technol ACM 119–122

  15. Malik S, Laszlo J (2004) Visual touchpad: a two-handed gestural input device. 289–296

  16. Malima A, Qzgur E, Cetin M (2006) A fast algorithm for vision-based hand gesture recognition for robot control. Proc 14th IEEE Conf Signal Process Commum Appl 1–4

  17. Mo Z, Lewis JP, Neumann U (2005) Smart Canvas:a gesture-driven intelligent drawing desk system. Int Conf Intell User Interf. January 10-13, 2005, San Diego, California, Usa DBLP 239–243

  18. Morshidi M, Tjahjadi T (2014) Gravity optimised particle filter for hand tracking. Pattern Recogn 47(1):194–207

    Article  Google Scholar 

  19. Núñez JC, Cabido R, Pantrigo JJ, Montemayor AS, Vélez JF (2018) Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recogn 76:80–94

    Article  Google Scholar 

  20. O'Hagan R, Zelinsky A (2000) Visual gesture interfaces for virtual environments. User Interface Conference, 2000. AUIC 2000. First Australasian IEEE 73–80

  21. Oka K, Sato Y, Koike H (2002) Real-time fingertip tracking and gesture recognition. Comput Graphics Appl IEEE 22(6):64–71

    Article  Google Scholar 

  22. Oka K, Sato Y, Koike H (2002) Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems. 情報処理学会論文誌コンピュータビジョンとイメージメディア(cvim) 44.568:25–32

  23. Pavlovic VI, Sharma R, Huang TS (1997) Visual interpretation of hand gestures for human-computer interaction: a review. IEEE Trans Pattern Anal Mach Intell 19(7):677–695

    Article  Google Scholar 

  24. Premaratne P, Yang S, Vial P, Ifthikar Z (2017) Centroid tracking based dynamic hand gesture recognition using discrete hidden Markov models. Neurocomputing 228:79–83

    Article  Google Scholar 

  25. Sagayam KM, Hemanth DJ (2016) Hand posture and gesture recognition techniques for virtual reality applications: a survey. Virtual Reality 21(2):1–17

    Google Scholar 

  26. Segen J, Kumar S (1998) Gesture VR: vision-based 3D hand interace for spatial interaction. ACM International Conference on Multimedia '98, Bristol, England, September DBLP 455–464

  27. Segen J, Kumar S (1998) Human-computer interactions using gesture recognition and 3D hand tracking. Proc IEEE Int Conf Image Process. 188–192

  28. Suau X et al (2014) Real-time fingertip localization conditioned on hand gesture classification. Image Vis Comput 32.8:522–532

    Article  Google Scholar 

  29. Tsironi E et al (2017) An analysis of convolutional long-short term memory recurrent neural networks for gesture recognition. Neurocomputing 268

  30. Hardenberg C Von (2001) Bare-hand human-computer interaction. The Workshop on Perceptive User Interfaces ACM 1–8

  31. Wu X et al (2015) Depth image-based hand tracking in complex scene. Optik Int J Light Electron Opt 126.20:2757–2763

    Article  Google Scholar 

  32. Yan C, Xie H, Chen J, Zha Z, Hao X, Zhang Y, Dai Q (2018) A fast Uyghur text detector for complex background images. IEEE Trans Multimed 20(12):3389–3398

    Article  Google Scholar 

  33. Yan C, Li L, Zhang C, Liu B, Zhang Y, Dai Q (2019) Cross-modality bridging and knowledge transferring for image understanding. IEEE Trans Multimed 1–1

  34. Yan C, Li Z, Zhang Y, Qin P, Ji X, Dai Q (2019) Depth image denoising using nuclear norm and learning graph model. IEEE Trans Multimed

  35. Yan C, Tu Y, Wang X, Zhang Y, Hao X, Zhang Y, Dai Q (2019) STAT: spatial-temporal attention mechanism for video captioning. IEEE Trans Multimed

  36. Zhang X et al (2015) Robust hand tracking via novel multi-cue integration. Neurocomputing 157:296–305

    Article  Google Scholar 

  37. Zhou Y, Jiang G, Lin Y (2015) A novel finger and hand pose estimation technique for real-time hand gesture recognition. Pattern Recogn 49.C:102–114

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Metrology and Measurement Engineering, China Jiliang University, Hangzhou, 310018, China

    Songxiao Cao

  2. State Key Laboratory of Fluid Power and Mechatronic System, Zhejiang University, Hangzhou, 310027, China

    Xuanyin Wang

Authors
  1. Songxiao Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xuanyin Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Songxiao Cao implemented the core algorithm, designed all the experiments, addressed the resulting data and drafted the manuscript. Xuanyin Wang participated in the design and construction of the system and helped draft the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Songxiao Cao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, S., Wang, X. Real-time dynamic gesture recognition and hand servo tracking using PTZ camera. Multimed Tools Appl 78, 27403–27424 (2019). https://doi.org/10.1007/s11042-019-07869-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07869-7

Keywords