Skip to main content

Advertisement

SpringerLink
Log in

Spatio-temporal 3D surface matching for hand gesture recognition using ICP algorithm

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

With emerging new applications like virtual reality, different algorithms for human action and gesture recognition have been proposed. In this paper, a new method for the recognition of moving hand gestures is presented. The proposed algorithm is based on the representation of hand motion as spatio-temporal 3D surfaces. Then, 3D surface matching is used to recognize the hand gesture. To form the spatio-temporal 3D surface of hand motion, we first apply the necessary preprocessing to video frames and extract hand contours. Then, by normalizing and overlapping hand contours in different frames, we construct spatio-temporal 3D surface of the hand gesture. To recognize hand gesture, we match the input 3D surface with surfaces in the database. For this purpose, we utilize ICP algorithm to find and compensate for 3D transformation between surfaces as well as the similarity measure between them. In real-world applications, hand motion is continuous and results in a sequence of disjointed hand gestures, which is called continuous hand gesture. To recognize continuous hand gestures, we propose an algorithm which first estimates probable disjointed gestures in the continuous gesture and then divides iteratively continuous gestures to true disjointed gestures. Finally, by applying a robust algorithm, the continuous gesture is recognized. We tested the proposed algorithm with hand gestures of American sign language and results showed the recognition rate of 94 % for disjointed gestures and 93.9 % for continuous gestures. The experimental results showed the efficiency of the proposed algorithm for hand gestures with noise as well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Derpanis, K.G.: A review of vision-based hand gestures. Internal Report, Department of Computer Science, York University, 2004. Online: http://www.cvr.yorku.ca/members/gradstudents/kosta/publications/file_Gesture_review.pdf

  2. Laviola Jr., J.J.: A survey of hand posture and gesture recognition techniques and technology. Tech. Rep. CS-99-11, Department of Computer Science, Brown University, June 1999. Online: www.cs.brown.edu/research/pubs/techreports/reports/CS-99-11.html

  3. Mitra, S., Acharya, T.: Gesture recognition: a survey. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 37(3), 311–324 (2007)

    Article  Google Scholar 

  4. Lin, Y.H., Chenb, C.H.: Template matching using the parametric template vector with translation, rotation and scale invariance. Pattern Recognit. 41(7), 2413–2421 (2008)

    Article  MATH  Google Scholar 

  5. Bhuyan, M.K., Ghosh, D., Bora, P.K.: Feature extraction from 2D gesture trajectory in dynamic hand gesture recognition. In: Proceedings IEEE Conference on Cybernetics and Intelligent Systems, pp. 1–6, June 7–9 (2006)

  6. Liu, N., Lovell, B.C.: Hand gesture extraction by active shape models. In: Proceedings of the Digital Image Computing: Techniques and Applications (DICTA’05), p. 10 (2005)

  7. Trucco, E., Verri, A.: Introductory Techniques for 3-D Computer Vision. Prentice Hall, New Jersey (1998)

    Google Scholar 

  8. Karami, A., Zanj, B., Kiani Sarkaleh, A.: Persian sign language (PSL) recognition using wavelet transform and neural networks. Expert Syst. Appl. 38(3), 2661–2667 (2011)

    Article  Google Scholar 

  9. Chung, W.K., Wu, X., Xu, Y.: A real time hand gesture recognition based on Haar wavelet representation. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics(ROBIO), Bangkok, pp. 336–341, (Feb. 2009)

  10. Huang, D.Y., Hu, W.C., Chang, S.H.: Vision-based hand gesture recognition using PCA+Gabor filters and SVM. In: Proceedings of the Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP09), Kyoto, pp. 1–4, (Sept. 2009)

  11. Li, H., Duan, H.B., Zhang, X.Y.: A novel image template matching based on particle filtering optimization. Pattern Recognit. Lett. Meta-heuristic Intell. Based Image Process. 31(13), 1825–1832 (2010)

    Article  Google Scholar 

  12. Sturman, D.J.: Whole-hand input. Ph.D dissertation, Massachusetts Institute of Technology, 1992. Online: http://xenia.media.mit.edu/djs/thesis.ftp.html

  13. Watson, R.: A survey of gesture recognition techniques. Technical Report, TCD-CS-93-11, Department of Computer Science, Trinity College Dublin, 1993. Online: http://www.tara.tcd.ie/jspui/handle/2262/12658

  14. Trevor, D.J., Pentland, A.P.: Recognition of space-time gestures using a distributed representation. MIT Media Laboratory Vision and Modeling Group Technical Report, No. 197, 1993. Online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.54.743&rep=rep1&type=pdf

  15. Zelinsky, A., Heinzmann, J.: Real-time visual recognition of facial gestures for human-computer interaction. In: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, pp. 351–356, Oct. 14–16 (1996)

  16. Hsieh, C.C., Liou, D.H., Cheng, Y.M., Cheng, F.C.: Robust visual mouse by motion history image. In: Proceedings of the IEEE International Conference on System Science and, Engineering (ICSSE2010), pp. 161–166, July 1–3 (2010)

  17. Rubine, D.: Specifying gestures by example. In: Proceedings of the SIGGRAPH, ACM Press, pp. 329–337 (1991)

  18. Wexelblat, A.: An approach to natural gesture in virtual environments. ACM Trans. Comput.-Hum. Interact. 2(3), 179–200 (1995)

    Google Scholar 

  19. Elmezain, M., Al-Hamadi, A., Pathan, S.S., Michaelis, B.: Spatio-temporal feature extraction-based hand gesture recognition for isolated american sign language and arabic numbers. In: Proceedings of the 6th International Symposium on Image and Signal Processing and Analysis(ISPA), pp. 254–259, Sept. 16–18 (2009)

  20. Wang, X., Xia, M., Cai, H., Gao, Y., Cattani, C.: Hidden-Markov-models-based dynamic hand gesture recognition. Math. Probl. Eng. 2012, 1–11 (2012)

    MATH  MathSciNet  Google Scholar 

  21. Heap, A.J., Samaria, F.: Real-time hand tracking and gesture recognition using smart snakes. In: Proceedings of the Interface to Real and Virtual Worlds, Montpellier, June (1995)

  22. Ezzati Chahar Ghaleh, V., Behrad, A.: Lip contour extraction using RGB color space and fuzzy C-means clustering. In: Proceedings of the IEEE SMC UK &RI 8th Conference on Cybernetic Systems, pp. 1–4, Sept. 1–2 (2009)

  23. Moses, Y., Reynard, D., Blake, A.: Determining facial expressions in real time. In: Proceedings of the Fifth International Conference on computer vision (ICCV95), pp. 296–301, (June 1995)

  24. Sukno, F.M., Ordas, S., Butakoff, C., Cruz, S., Frangi, A.F.: Active shape models with invariant optimal features: application to facial analysis. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1105–1117 (2007)

    Article  Google Scholar 

  25. Collins, T.: Analyzing video sequences using the spatio-temporal volume. MSc. Inform. Res. Rev. (2004)

  26. Konrad, J., Ristivojevic, M.: Joint space-time image sequence segmentation: object tunnels and occlusion volumes. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2004) 3, 9–12 (2004)

  27. Zhong, Y., Stevens, M.: Action recognition in spatiotemporal volume. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR2010), pp. 25–30, June 13–18 (2010)

  28. Kim, T.K., Cipolla, R.: Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Trans. Pattern Anal. Mach. Intell. 31(8), 1415–1428 (2009)

    Article  Google Scholar 

  29. Li, H., Greenspan, M.: Model-based segmentation and recognition of dynamic gestures in continuous video streams. Pattern Recognit. 44(8), 1614–1628 (2011)

    Google Scholar 

  30. Li, H., Greenspan, M.: Multi-scale gesture recognition from time-varying contours. In: Proceedings of the International Conference on Computer Vision, pp. 236–243, Oct. 17–21 (2005)

  31. Li, H., Greenspan, M.: Continuous time-varying gesture segmentation by dynamic time warping of compound gesture models. In: Proceedings of the International Workshop on Human Activity Recognition and Modeling (HARAM), pp. 35–42 (2005)

  32. Li, H., Greenspan, M.: Segmentation and recognition of continuous gestures. In: Proceedings of the International Conference on Image Processing, pp. I365–I368, Sept. 16–Oct. 19 (2007)

  33. Zhang, D., Lu, G.: Review of shape representation and description techniques. Pattern Recognit. 37(1), 1–19 (2004)

    Article  MATH  Google Scholar 

  34. Gruen, A., Akcea, D.: Least squares 3D surface and curve matching. ISPRS J. Photogramm. Remote Sending 59(3), 151–174 (2005)

    Article  Google Scholar 

  35. Sen, W., Yang, W., Miao, J., Gu, X., Samaras, D.: 3D surface matching and recognition using conformal geometry. In: Proceedings of the IEEE Computer Society Conference on computer Vision and, Pattern Recognition (CVPR2006), pp. 2453–2460 (2006)

  36. Liu, Y.: Improving ICP with easy implementation for free-form surface matching. Pattern Recognit. 37(2), 211–226 (2004)

    Article  MATH  Google Scholar 

  37. Jones, M.J., Rehg, J.M.: Statistical color models with application to skin detection. Int. J. Comput. Vis. 46(1), 81–96 (1999)

    Article  Google Scholar 

  38. Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  39. Sharp, G.C., Lee, S.W., Wehe, D.K.: ICP registration using invariant features. IEEE Trans. Pattern Anal. Mach. Intell. 24(1), 90–102 (2002)

    Article  Google Scholar 

  40. Almhdie, A., Lger, C., Deriche, M., Lde, R.: 3D registration using a new implementation of the ICP algorithm based on a comprehensive lookup matrix: application to medical imaging. Pattern Recognit. Lett. 28(12), 1523–1533 (2007)

    Article  Google Scholar 

  41. Ezra, E., Sharir, M., Efrat, A.: On the performance of the ICP algorithm. Comput. Geom. 41(12), 7793 (2008)

    MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engineering Department, Faculty of Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Saeed Nasri & Farbod Razzazi

  2. Electrical Engineering Department, Faculty of Engineering, Shahed University, Tehran, Iran

    Alireza Behrad

Authors
  1. Saeed Nasri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alireza Behrad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farbod Razzazi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeed Nasri.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nasri, S., Behrad, A. & Razzazi, F. Spatio-temporal 3D surface matching for hand gesture recognition using ICP algorithm. SIViP 9, 1205–1220 (2015). https://doi.org/10.1007/s11760-013-0558-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-013-0558-7

Keywords

Search

Navigation