Skip to main content
SpringerLink
Log in

Real-time recognition of medial structures within hand postures through Eigen-space and geometric skeletal shape features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Skeletons provide landmark points that preserve implicit strokes or medial structure within a shape for compact representation. Though, hand posture shapes are usually recognized by region or contour representations, some applications may only be interested in the recognition of medial structures within the postures rather than their exact outlines and regions. Proposed work identifies several unique medial structures formed by a set of both one and two-handed postures and demonstrates their pure skeletal recognition in real-time. Existing skeleton-based recognition schemes apply the complex segmental processing on underlying skeleton and rely on contour information which is not suitable for fast recognition of medial structures. Presented work applies intuitive Eigen-space based Principal Components of Symbolic Structure (PCSS) and geometric Equi-Polar Signature (EPS) features to accomplish the recognition task. Both PCV and EPS process the skeleton globally without sections without associating contour information. Recognition accuracy up to 94% is obtained on a 22 posture dataset comprising of 10,560 depth frames with 480 samples for each posture. Depth sensor based acquisition is employed to meet the real-time requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Bai X, Latecki LJ (2008) Path similarity skeleton graph matching. IEEE Trans Pattern Anal Mach Intell 30(7):1282–1292

    Article  Google Scholar 

  2. Barkoky A, Charkari NM (2011) Static hand gesture recognition of Persian sign numbers using thinning method. In: IEEE International Conference on Multimedia Technology (ICMT), pp. 6548–6551

  3. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  4. Bourennane S, Fossati C (2012) Comparison of shape descriptors for hand posture recognition in video. SIViP 6(1):147–157

    Article  Google Scholar 

  5. Chang CC, Wu TC (1995) An exact match retrieval scheme based upon principal component analysis. Pattern Recogn Lett 16(5):465–470

    Article  Google Scholar 

  6. Chen C, Liu M, Zhang B, Han J, Jiang J, Liu H (2016) 3D Action Recognition Using Multi-temporal Depth Motion Maps and Fisher Vector. IJCAI:3331–3337

  7. Cobos S, Ferre M, Uran S, Ortego J, Pena C (2008) Efficient human hand kinematics for manipulation tasks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2008. IROS 2008, pp 2246–2251

  8. Deng W, Iyengar SS, Brener NE (2000) A fast parallel thinning algorithm for the binary image skeletonization. Int J High Perform Comput Appl 14(1):65–81

    Article  Google Scholar 

  9. Dominio F, Donadeo M, Zanuttigh P (2014) Combining multiple depth-based descriptors for hand gesture recognition. Pattern Recogn Lett 50:101–111

    Article  Google Scholar 

  10. El-Khoury S, Li M, Billard A (2013) On the generation of a variety of grasps. Robot Auton Syst 61(12):1335–1349

    Article  Google Scholar 

  11. Fujimura K, Liu X (2006) Sign recognition using depth image streams. In: IEEE 7th International Conference on Automatic Face and Gesture Recognition (FGR), pp. 381–386

  12. Goh WB (2008) Strategies for shape matching using skeletons. Comput Vis Image Underst 110(3):326–345

    Article  Google Scholar 

  13. Gonzalez RC, Woods RE, Eddins SL (2012) Digital image processing using MATLAB, 2nd edn. Tata McGraw-Hill, New Delhi

    Google Scholar 

  14. Guru DS, Punitha P (2004) An invariant scheme for exact match retrieval of symbolic images based upon principal component analysis. Pattern Recogn Lett 25(1):73–86

    Article  Google Scholar 

  15. Guru DS, Punitha P, Nagabhushan P (2003) Archival and retrieval of symbolic images: an invariant scheme based on triangular spatial relationship. Pattern Recogn Lett 24(14):2397–2408

    Article  MATH  Google Scholar 

  16. Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor: a review. IEEE Trans Cybern 43(5):1318–1334

    Article  Google Scholar 

  17. Jiang M, Kong J, Bebis G, Huo H (2015) Informative joints based human action recognition using skeleton contexts. Signal Process Image Commun 33:29–40

    Article  Google Scholar 

  18. Kimia BB, Tannenbaum AR, Zucker SW (1995) Shapes, shocks, and deformations I: the components of two-dimensional shape and the reaction-diffusion space. Int J Comput Vis 15(3):189–224

    Article  Google Scholar 

  19. Kirac F, Kara YE, Akarun L (2014) Hierarchically constrained 3D hand pose estimation using regression forests from single frame depth data. Pattern Recogn Lett 50:91–100

    Article  Google Scholar 

  20. Krinidis S, Krinidis M (2014) Skeletonization based on angle maps. Pattern Anal Applic 17(3):517–528

    Article  MathSciNet  Google Scholar 

  21. Lam L, Lee SW, Suen CY (1992) Thinning methodologies-a comprehensive survey. IEEE Trans Pattern Anal Mach Intell 14(9):869–885

    Article  Google Scholar 

  22. Lee AJ, Chiu HP (2003) 2D Z-string: a new spatial knowledge representation for image databases. Pattern Recogn Lett 24(16):3015–3026

    Article  Google Scholar 

  23. Li X., An JH, Min JH, Hong KS (2011) Hand gesture recognition by stereo camera using the thinning method. In: IEEE International Conference on Multimedia Technology (ICMT), pp. 3077–3080

  24. Liu L, Xing J, Ai H, & Ruan X (2012). Hand posture recognition using finger geometric feature. In IEEE 21st International Conference on Pattern Recognition (ICPR), pp. 565–568

  25. Ma J, Choi S (2014) Kinematic skeleton extraction from 3D articulated models. Comput Aided Des 46:221–226

    Article  Google Scholar 

  26. Narayana M, Sandeep VM, Kulkarni S (2011) Skeleton based signatures for content based image retrieval. Int J Comput Appl 23(7):29–34

    Google Scholar 

  27. Pedersoli F, Benini S, Adami N, Leonardi R (2014) XKin: an open source framework for hand pose and gesture recognition using kinect. Vis Comput 30(10):1107–1122

    Article  Google Scholar 

  28. Plouffe G, Cretu AM (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Meas 65(2):305–316

    Article  Google Scholar 

  29. Pugeault N, and Bowden R (2011) Spelling it out: Real-time asl fingerspelling recognition. In IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 1114–1119

  30. Qian C, Sun X, Wei Y, Tang X, Sun J (2014) Realtime and robust hand tracking from depth. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p 1106–1113

  31. Rempel D, Camilleri MJ, Lee DL (2014) The design of hand gestures for human–computer interaction: lessons from sign language interpreters. Int J Hum Comput Stud 72(10):728–735

    Article  Google Scholar 

  32. Ren Z, Yuan J, Meng J, Zhang Z (2013) Robust part-based hand gesture recognition using kinect sensor. IEEE Trans Multimed 15(5):1110–1120

    Article  Google Scholar 

  33. Roberts, L., Singhal, G., Kaliki, R. (2011) : Slip detection and grip adjustment using optical tracking in prosthetic hands. In Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC, pp. 2929–2932

  34. Rubner Y, Tomasi C, Guibas LJ (2000) The earth mover’s distanceas a metric for image retrieval. Int J Comput Vis 40:99–121

    Article  MATH  Google Scholar 

  35. Sebastian TB, Kimia BB (2005) Curves vs. skeletons in object recognition. Signal Process 85(2):247–263

    Article  MATH  Google Scholar 

  36. Shotton J, Sharp T, Kipman A, Fitzgibbon A, Finocchio M, Blake A, Cook M, Moore R (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56(1):116–124

    Article  Google Scholar 

  37. Soffer A, Samet H (1999) Two data organizations for storing symbolic images in a relational database system. In: Database semantics, Springer, US, pp 435–456

  38. Stergiopoulou E, Papamarkos N (2009) Hand gesture recognition using a neural network shape fitting technique. Eng Appl Artif Intell 22(8):1141–1158

    Article  Google Scholar 

  39. Su Z, Cao Z, Wang Y, Zhen X (2011) Identification of unreliable segments to improve skeletonization of handwriting images. Pattern Anal Applic 14(1):77–86

    Article  MathSciNet  Google Scholar 

  40. Suau X, Alcoverro M, López-Méndez A, Ruiz-Hidalgo J, Casas JR (2014) Real-time fingertip localization conditioned on hand gesture classification. Image Vis Comput 32(8):522–532

    Article  Google Scholar 

  41. Tang D, Yu TH, Kim TK (2013) Real-time articulated hand pose estimation using semi-supervised transductive regression forests. In: IEEE International Conference on Computer Vision (ICCV), pp. 3224–3231

  42. Torsello A, Hancock ER (2003) Curvature dependent skeletonization. In: Image analysis. Springer, Berlin, pp. 200–207

    Chapter  Google Scholar 

  43. Wang J, Liu Z, Chorowski J, Chen Z, Wu Y (2012) Robust 3d action recognition with random occupancy patterns. In: Computer vision–ECCV 2012. Springer, Berlin, pp. 872–885

    Chapter  Google Scholar 

  44. Wang X, Yang W, Peng H, Wang G (2013) Shape-aware skeletal deformation for 2D characters. Vis Comput 29:545–553

    Article  Google Scholar 

  45. Wang X, Wang R, Zhou F (2014) Fingertips detection and hand tracking based on curve fitting. In IEEE 7th International Congress on Image and Signal Processing (CISP), pp. 99–103

  46. Wu D and Shao L (2014) Leveraging hierarchical parametric networks for skeletal joints based action segmentation and recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 724–731

  47. Wu D, Pigou L, Kindermans PJ, Nam LE, Shao L, Dambre J, Odobez JM (2016) Deep dynamic neural networks for multimodal gesture segmentation and recognition. IEEE Trans Pattern Anal Mach Intell:1–16

  48. Xie J, Heng PA, Shah M (2008) Shape matching and modeling using skeletal context. Pattern Recogn 41(5):1756–1767

    Article  MATH  Google Scholar 

  49. Xie S., Liu J, Smith CD (2012) Curve skeleton-based shape representation and classification. In: IEEE 19th International Conference on Image Processing (ICIP), pp 529–532

  50. Yu D, Jin JS, Luo S, Lai W, Park M, Pham TD (2010) Shape analysis and recognition based on skeleton and morphological structure. In: Seventh International Conference on Computer Graphics, Imaging and Visualization (CGIV), pp. 118–123

  51. Yu M, Liu L, Shao L (2016) Structure-preserving binary representations for RGB-D action recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1651–1664

    Article  Google Scholar 

  52. Zhang TY, Suen CY (1984) A fast parallel algorithm for thinning digital patterns. Commun ACM 27(3):236–239

    Article  Google Scholar 

  53. Zhang B, Perina A, Murino V and Del Bue A (2015) Sparse representation classification with manifold constraints transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4557–4565

  54. Zhang B, Perina A, Li Z, Murino V, Liu J and Ji R (2016) Bounding multiple gaussians uncertainty with application to object tracking. International Journal of Computer Vision, pp1–16

Download references

Author information

Authors and Affiliations

  1. Discipline of Computer Science and Engineering, PDPM Indian Institute of Information Technology, Design and Manufacturing, Jabalpur, India

    Lalit Kane & Pritee Khanna

Authors
  1. Lalit Kane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pritee Khanna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pritee Khanna.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kane, L., Khanna, P. Real-time recognition of medial structures within hand postures through Eigen-space and geometric skeletal shape features. Multimed Tools Appl 76, 4571–4598 (2017). https://doi.org/10.1007/s11042-016-4173-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4173-9

Keywords

Search

Navigation