Skip to main content

Advertisement

SpringerLink
Log in

Hand gesture recognition using topological features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hand Gestures Recognition (HGR) is one of the main areas of research for Human Computer Interaction applications. Most existing approaches are based on local or geometrical properties of pixels. Still, there are some serious challenges on HGR methods such as sensitivity to rotation, scale, illumination, perturbation, and occlusion. In this paper, we study HGR from graph viewpoints. We introduce a set of meaningful shape features based on a graph constructed by Growing Neural Gas (GNG) algorithm. These features are constructed from topological properties of this graph. Graph properties in conserving topological features improve stability against different deformations, scale, and noise. We evaluate our method on NTU Hand Digits dataset with state-of-the-art methods. We also prepared a comprehensive dataset (SBU-1) for different hand gestures containing 2170 images. This dataset includes many possible deformations and variations and some articulations. Most of the existing datasets don’t capture these variations. We show the robustness of the algorithm to scale, rotation and noise, while preserving similar recognition rate in comparison with the state-of-the-art results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Bai X, Latecki LJ (2008) Path similarity skeleton graph matching. IEEE Trans Pattern Anal Mach Intell 30(7):1282–1292

    Article  Google Scholar 

  2. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  3. Bondy J, Murty U (2008) Graph theory, 2nd printing. Springer

  4. Bourke A, Obrien J, Lyons G (2007) Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm. Gait Posture 26(2):194–199

    Article  Google Scholar 

  5. Bretzner L, Laptev I, Lindeberg T (2002) Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering. In: Proceedings. Fifth IEEE international conference on automatic face and gesture recognition. IEEE, pp 423–428

  6. Chaudhary A, Raheja J, Das K, Raheja S (2011) A survey on hand gesture recognition in context of soft computing. In: International conference on computer science and information technology. Springer, pp 46–55

  7. Cheng H, Dai Z, Liu Z, Zhao Y (2016) An image-to-class dynamic time warping approach for both 3d static and trajectory hand gesture recognition. Pattern Recognit 55:137–147

    Article  Google Scholar 

  8. Corera S, Krishnarajah N (2011) Capturing hand gesture movement: a survey on tools techniques and logical considerations. In: Proceedings of chi sparks

  9. De Berg M, Van Kreveld M, Overmars M, Schwarzkopf OC (2000) Computational geometry. Springer, Berlin

    Book  MATH  Google Scholar 

  10. Dong C, Leu MC, Yin Z (2015) American sign language alphabet recognition using microsoft kinect. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–52

  11. Flórez F, García JM, García J, Hernández A (2002) Hand gesture recognition following the dynamics of a topology-preserving network. In: Proceedings. Fifth IEEE international conference on automatic face and gesture recognition. IEEE, pp 318–323

  12. Fritzke B et al (1995) A growing neural gas network learns topologies. In: Advances in neural information processing systems, vol 7, pp 625–632

  13. Fink O, Zio E, Weidmann U (2015) Novelty detection by multivariate kernel density estimation and growing neural gas algorithm. Mech Syst Signal Process 50:427–436

    Article  Google Scholar 

  14. Ge SS, Yang Y, Lee TH (2008) Hand gesture recognition and tracking based on distributed locally linear embedding. Image Vis Comput 26(12):1607–1620

    Article  Google Scholar 

  15. Hall ET et al (1959) The silent language, vol 3. Doubleday, New York

  16. Holdstein Y, Fischer A (2008) Three-dimensional surface reconstruction using meshing growing neural gas (MGNG). Vis Comput 24(4):295–302

    Article  Google Scholar 

  17. Karam M (2006) Phd thesis: a framework for research and design of gesture-based humancomputer interactions. PhD thesis, University of Southampton

  18. Kirac F, Kara YE, Akarun L (2014) Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data. Pattern Recogn Lett 50:91–100

    Article  Google Scholar 

  19. Kim TK, Cipolla R (2009) Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Trans Pattern Anal Mach Intell 31(8):1415–1428

    Article  Google Scholar 

  20. Klein HA (2012) The science of measurement: a historical survey, Courier Corporation, North Chelmsford

  21. Li S, Zhang H (2004) Floatboost learning and statistical face detection. IEEE Trans Pattern Anal Mach Intell 26(9):1112–1123

    Article  Google Scholar 

  22. Li Y, Wang X, Liu W, Feng B (2018) Deep attention network for joint hand gesture localization and recognition using static RGB-D images. Inf Sci 441:66–78

    Article  MathSciNet  Google Scholar 

  23. Liu K, Kehtarnavaz N (2016) Real-time robust vision-based hand gesture recognition using stereo images. J Real-Time Image Proc 11(1):201–209

    Article  Google Scholar 

  24. Marcel S, Bernier O, Viallet JE, Collobert D (2000) Hand gesture recognition using input-output hidden markov models. In: Automatic face and gesture recognition. IEEE, pp 456–461

  25. Maqueda AI, del-Blanco CR, Jaureguizar F, García N (2015) Human–computer interaction based on visual hand-gesture recognition using volumetric spatiograms of local binary patterns. Comput Vis Image Underst 141:126–37

    Article  Google Scholar 

  26. Martinetz T, Schulten K et al (1991) A “neuralgas” network learns topologies. University of Illinois at Urbana-Champaign

  27. Memo A, Zanuttigh P (2018) Head-mounted gesture controlled interface for human-computer interaction. Multimed Tools Appl 77(1):27–53

    Article  Google Scholar 

  28. Mittal A, Zisserman A, Torr PH (2011) Hand detection using multiple proposals. In: BMVC, pp 1–11

  29. Molina J, Pajuelo JA, Escudero-Vi nolo M, Bescós J, Martínez JM (2014) A natural and synthetic corpus for benchmarking of hand gesture recognition systems. Mach Vis Appl 25(4):943–954

    Article  Google Scholar 

  30. Munib Q, Habeeb M, Takruri B, Al-Malik HA (2007) American sign language (asl) recognition based on hough transform and neural networks. Exp Syst Appl 32(1):24–37

    Article  Google Scholar 

  31. Nagi J, Ducatelle F, Di Caro GA, Cireşan D, Meier U, Giusti A, Nagi F, Schmidhuber J, Gambardella LM (2011) Max-pooling convolutional neural networks for vision-based hand gesture recognition. In: IEEE international conference on signal and image processing applications (ICSIPA), vol 2011. IEEE, pp 342–347

  32. Nai W, Liu Y, Rempel D, Wang Y (2017) Fast hand posture classification using depth features extracted from random line segments. Pattern Recogn 65:1–10

    Article  Google Scholar 

  33. Orts-Escolano S, Garcia-Rodriguez J, Morell V, Cazorla M, Perez JA, Garcia-Garcia A (2016) 3d surface reconstruction of noisy point clouds using growing neural gas: 3d object/scene reconstruction. Neural Process Lett 43(2):401–423

    Article  Google Scholar 

  34. Pattanaworapan K, Chamnongthai K, Guo JM (2016) Signer-independence finger alphabet recognition using discrete wavelet transform and area level run lengths. J Vis Commun Image Represent 38:658–677

    Article  Google Scholar 

  35. Peng B, Zhang L, Zhang D (2013) A survey of graph theoretical approaches to image segmentation. Pattern Recogn 46(3):1020–1038

    Article  Google Scholar 

  36. Phung SL, Bouzerdoum A, Chai D (2005) Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans Pattern Anal Mach Intell 27(1):148–154

    Article  Google Scholar 

  37. Plouffe G, Cretu AM (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Meas 65(2):305–16

    Article  Google Scholar 

  38. Priyal SP, Bora PK (2013) A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments. Pattern Recogn 46(8):2202–2219

    Article  MATH  Google Scholar 

  39. Pugeault N, Bowden R (2011) Spelling it out: realtime asl fingerspelling recognition. In: IEEE international conference on computer vision workshops (ICCV Workshops), 2011. IEEE, pp 1114--1119

  40. Rautaray SS, Agrawal A (2015) Vision based hand gesture recognition for human computer interaction: a survey. Artif Intell Rev 43(1):1–54

    Article  Google Scholar 

  41. Ren Z, Yuan J, Zhang Z (2011) Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: Proceedings of the 19th ACM international conference on Multimedia. ACM, pp 1093–1096

  42. Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimedia 15(5):1110–20

    Article  Google Scholar 

  43. Siddiqi K, Shokoufandeh A, Dickinson SJ, Zucker SW (1999) Shock graphs and shape matching. Int J Comput Vis 35(1):13–32

    Article  Google Scholar 

  44. Stergiopoulou E, Papamarkos N (2009) Hand gesture recognition using a neural network shape fitting technique. Eng Appl Artif Intell 22(8):1141–1158

    Article  Google Scholar 

  45. Stergiopoulou E, Sgouropoulos K, Nikolaou N, Papamarkos N, Mitianoudis N (2014) Real time hand detection in a complex background. Eng Appl Artif Intell 35:54–70

    Article  Google Scholar 

  46. Sun Q, Liu H, Harada T (2017) Online growing neural gas for anomaly detection in changing surveillance scenes. Pattern Recogn 64:187–201

    Article  Google Scholar 

  47. Viola P, Jones M (2002) Fast and robust classification using asymmetric adaboost and a detector cascade. In: Advances in neural information processing systems, pp 1311–1318

  48. Wang C, Liu Z, Zhu M, Zhao J, Chan SC (2017) A hand gesture recognition system based on canonical superpixel-graph. Signal Process Image Commun 58:87–98

    Article  Google Scholar 

  49. Wang Y, Yang R (2013) Real-time hand posture recognition based on hand dominant line using kinect. In: IEEE international conference multimedia and expo workshops (ICMEW). IEEE, pp 1–4

  50. Wobbrock JO, Wilson AD, Li Y (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology. ACM, pp 159–168

  51. Wu Y, Huang TS (2001) Hand modeling, analysis and recognition. IEEE Signal Process Mag 18(3):51–60

    Article  Google Scholar 

  52. Xie S, Pan J (2011) Hand detection using robust color correction and gaussian mixture model. In: Sixth international conference on image and graphics (ICIG). IEEE, pp 553–557

  53. Zhang C, Yang X, Tian Y (2013) Histogram of 3d facets: a characteristic descriptor for hand gesture recognition. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG). IEEE, pp 1–8

  54. Zhang H, Fritts JE, Goldman SA (2008) Image segmentation evaluation: a survey of unsupervised methods. Comput Vis Image Underst 110(2):260–280

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Shahid Beheshti University, G.C., Tehran, Iran

    Narges Mirehi, Maryam Tahmasbi & Alireza Tavakoli Targhi

Authors
  1. Narges Mirehi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maryam Tahmasbi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alireza Tavakoli Targhi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maryam Tahmasbi.

Ethics declarations

Conflict of Interest

Narges Mirehi declares that she has no conflict of interest. Maryam Tahmasbi declares that she has no conflict of interest. Alireza Tavakoli Targhi declares that he has no conflict of interest. The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mirehi, N., Tahmasbi, M. & Targhi, A.T. Hand gesture recognition using topological features. Multimed Tools Appl 78, 13361–13386 (2019). https://doi.org/10.1007/s11042-019-7269-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7269-1

Keywords

Search

Navigation