Skip to main content
SpringerLink
Log in

Real-time robust vision-based hand gesture recognition using stereo images

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper presents a real-time and robust approach to recognize two types of gestures consisting of seven motional gestures and six finger spelling gestures. This approach utilizes stereo images captured by a stereo webcam to achieve robust recognition under realistic lighting conditions and in various backgrounds. It incorporates several existing computationally efficient techniques and introduces a rule-based approach to merge the information from a pair of stereo images leading to an improved hand detection compared to using single images. The results obtained indicate that high recognition rates under realistic conditions are obtained in real-time on PC platforms at the rate of 30 frames per second. It is shown that its outcome is comparable to two existing approaches while it is computationally more efficient than these approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Zimmerman, T., Lanier, J., Blanchard, C., Bryson, S., Harvill, Y.: A hand gesture interface device. In: Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface, vol. 18, no. 4, pp. 189–192. Toronto, Canada (1987)

  2. Keskin, C., Kirac, F., Kara, Y., Akarun, L.: Real time hand pose estimation using depth sensors. In: Proceedings of IEEE International Conference on Computer Vision Workshops, pp. 1228–1234. Barcelona, Spain (2011)

  3. Ren, Z., Meng, J., Yuan, J., Zhang Z.: Robust hand gesture recognition with kinect sensor. In: Proceedings of the ACM International Conference on Multimedia, pp. 759–760. Scottsdale, AZ (2011)

  4. Van Den Bergh, M., Van Gool, L.: Combining RGB and ToF cameras for real-time 3D hand gesture interaction. IEEE Workshop on Applications of Computer Vision, pp. 66–72. Kona, HI (2011)

  5. Rodriguez, S., Picon, A., Villodas, A.: Robust vision-based hand tracking using single camera for ubiquitous 3D gesture interaction. In: Proceedings of IEEE Symposium on 3D User Interfaces, pp. 135–136. Waltham, MA (2010)

  6. Gorce, D., Fleet, D., Paragios, N.: Model-based 3D hand pose estimation from monocular video. IEEE Trans. Pattern Anal. Mach. Intell. 33(9), 1793–1805 (2011)

    Article  Google Scholar 

  7. Ren, Y., Zhang, F.: Hand gesture recognition based on MEB-SVM. In: Proceedings of International Conference on Embedded Software and Systems, pp. 344–349. Hangzhou, China (2009)

  8. Triesch, J., von der Malsburg, C.: Robust classification of hand postures against complex backgrounds. Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pp. 170–175. Killington, VT (1996)

  9. Quek, F., Zhao, M.: Inductive learning in hand pose recognition. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, pp. 78–83. Killington, VT (1996)

  10. Park, A., Yun, S., Kim, J., Min, S., Jung, K.: Real-time vision based Korean finger spelling recognition system. Int. J. Electr. Comput. Eng. 4, 110–115 (2009)

    Google Scholar 

  11. Murase, H., Nayar, S.: Visual learning and recognition of 3D objects from appearance. Int. J. Comput. Vis. 14, 5–24 (1995)

    Article  Google Scholar 

  12. Bradski, G., Davis, J.: Motion segmentation and pose recognition with motion history gradients. In: Proceedings of IEEE Workshop on Applications of Computer Vision, pp. 238–244. Palm Springs, CA (2000)

  13. Lee, D., Hong, K.: A Hand gesture recognition system based on difference image entropy. In: Proceedings of the IEEE International Conference on Advanced Information Management and Service, pp. 410–413. Suwon, South Korea (2010)

  14. Lee, D., Hong, K.: Game interface using hand gesture recognition. In: Proceedings of the IEEE International Conference on Computer Sciences and Convergence Information Technology, pp. 1092–1097. Suwon, South Korea (2010)

  15. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)

    Article  MATH  Google Scholar 

  16. Rahman, M., Ren, J., Kehtarnavaz, N.: Real-time implementation of robust face detection on mobile platforms. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1353–1356. Taipei, Taiwan (2009)

  17. Liu, K., Du, Q., Yang, H., Ma, B.: Optical flow and principal component analysis-based motion detection in outdoor videos. EURASIP Journal on Advances in Signal Processing 680623, 2010

  18. Bruhn, A., Weichert, J., Schnorr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vis. 61(3), 211–231 (2005)

    Article  Google Scholar 

  19. Bradski, G.: Computer video face tracking for use in a perceptual user interface. Intel Technol. J. Q2, (1998)

  20. Heckbert, P.: A Seed Fill Algorithm From Graphics Gems. Academic Press, New York (1990)

    Google Scholar 

  21. Rahman, M., Kehtarnavaz, N., Ren, J.: A hybrid face detection approach for real-time deployment on mobile devices. Proceedings of IEEE Conference on Image Processing, pp. 3233–3236. Cairo, Egypt (2009)

  22. Perez, P., Blake, A., Gangnet, M.: Jetstream: probabilistic contour extraction with particles. In: Proceedings of International Conference on Computer Vision, vol. 2, pp. 524–531. Vancouver, Canada (2001)

  23. Soille, P.: Morphological Image Analysis: Principles and Applications. Springer, New York (2003)

    Google Scholar 

  24. Liu, L., Zhang, S., Zhang, Y., Ye, X.: Human contour extraction using level set. In: Proceedings of IEEE International Conference on Computer and Information Technology, pp. 608–613. Shanghai, China (2005)

  25. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1(4), 321–331 (1998)

    Article  Google Scholar 

  26. Mahotra, S., Patllola, C., Kehtarnavaz, N.: Real-time computation of disparity for hand-pair gesture recognition using video stereo images. J. Real-Time Image Process. 7, 257–266 (2012)

    Article  Google Scholar 

  27. Wang, L., Liao, M., Gong, M., Yang , R., Nister, D.: High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: Proceedings of IEEE International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 798–805. Chapel Hill, NC (2006)

  28. Senin, P.: Dynamic Time warping algorithm review. In: Technical Report. Information and Computer Science Department, University of Hawaii at Manoa (2008)

  29. Li, X., An, J., Min, J., Hong, K.: Hand gesture recognition by stereo camera using the thinning method. In: Proceedings of IEEE International Conference on Multimedia Technology, pp. 3077-3080. Hangzhou, China (2011)

Download references

Author information

Authors and Affiliations

  1. University of Texas at Dallas, Richardson, TX, USA

    Kui Liu & Nasser Kehtarnavaz

Authors
  1. Kui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nasser Kehtarnavaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kui Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, K., Kehtarnavaz, N. Real-time robust vision-based hand gesture recognition using stereo images. J Real-Time Image Proc 11, 201–209 (2016). https://doi.org/10.1007/s11554-013-0333-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0333-6

Keywords

Search

Navigation