Abstract
Hand gesture recognition is recently becoming one of the most attractive field of research in Pattern Recognition. In this paper, a skeleton-based approach is proposed for 3D hand gesture recognition. Specifically, we consider the sequential data of hand geometric configuration to capture the hand shape variation, and explore the temporal character of hand motion. 3D Hand gesture are represented as a set of relevant spatiotemporal motion trajectories of hand-parts in an Euclidean space. Trajectories are then interpreted as elements lying on Riemannian manifold of shape space to capture their shape variations and achieve gesture recognition using a linear SVM classifier.
The proposed approach is evaluated on a challenging hand gesture dataset containing 14 gestures, performed by 20 participants performing the same gesture with two different numbers of fingers. Experimental results show that our skeleton-based approach consistently achieves superior performance over a depth-based approach.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Kuznetsova, A., Leal-Taixé, L., Rosenhahn, B.: Real-time sign language recognition using a consumer depth camera. In: IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 83–90, December 2013
Wang, H., Wang, Q., Chen, X.: Hand posture recognition from disparity cost map. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012. LNCS, vol. 7725, pp. 722–733. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-37444-9_56
Ren, Z., Yuan, J., Zhang, Z.: Robust hand gesture recognition based on finger-earth mover’s distance with a commodity depth camera. In: ACM International Conference on Multimedia, MM 2011, pp. 1093–1096. ACM, New York (2011)
Cheng, H., Dai, Z., Liu, Z.: Image-to-class dynamic time warping for 3D hand gesture recognition. In: 2013 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6, July 2013
Pugeault, N., Bowden, R.: Spelling it out: real-time ASL fingerspelling recognition. In: IEEE Computer Vision Workshops (ICCV Workshops), pp. 1114–1119, November 2011
Dong, C., Leu, M.C., Yin, Z.: American sign language alphabet recognition using microsoft kinect. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 44–52, June 2015
Marin, G., Dominio, F., Zanuttigh, P.: Hand gesture recognition with leap motion and kinect devices. In: IEEE International Conference on Image Processing (ICIP), pp. 1565–1569 (2014)
Kurakin, A., Zhang, Z., Liu, Z.: A real time system for dynamic hand gesture recognition with a depth sensor. In: 20th European Signal Processing Conference (EUSIPCO), pp. 1975–1979, August 2012
Zhang, C., Yang, X., Tian, Y.: Histogram of 3D facets: a characteristic descriptor for hand gesture recognition. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–8, April 2013
Escalera, S., et al.: ChaLearn looking at people challenge 2014: dataset and results. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 459–473. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_32
Monnier, C., German, S., Ost, A.: A multi-scale boosted detector for efficient and robust gesture recognition. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8925, pp. 491–502. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16178-5_34
Neverova, N., Wolf, C., Taylor, G.W., Nebout, F.: ModDrop: adaptive multi-modal gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell., April 2016
Ohn-Bar, E., Trivedi, M.M.: Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations. IEEE Trans. Intell. Transp. Syst. 15(6), 2368–2377 (2014)
De Smedt, Q., Wannous, H., Vandeborre, J.P.: Skeleton-based dynamic hand gesture recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2016
Lee, C.S., Elgammal, A.M.: Modeling view and posture manifolds for tracking. In: IEEE International Conference on Computer Vision, pp. 1–8 (2007)
Lui, Y.M.: Advances in matrix manifolds for computer vision. Image Vis. Comput. 30, 380–388 (2012)
Harandi, M.T., Sanderson, C., Shirazi, S., Lovell, B.C.: Kernel analysis on grassmann manifolds for action recognition. Pattern Recogn. Lett. 34, 1906–1915 (2013)
Vemulapalli, R., Arrate, F., Chellappa, R.: Human action recognition by representing 3D skeletons as points in a lie group. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, pp. 588–595. IEEE Computer Society, Washington, DC (2014)
Slama, R., Wannous, H., Daoudi, M., Srivastava, A.: Accurate 3D action recognition using learning on the Grassmann manifold. Pattern Recogn. 48(2), 556–567 (2015)
Slama, R., Wannous, H., Daoudi, M.: 3D human motion analysis framework for shape similarity and retrieval. Image Vis. Comput. 32(2), 131–154 (2014)
Joshi, S.H., Klassen, E., Srivastava, A., Jermyn, I.: A novel representation for Riemannian analysis of elastic curves in \(R^n\). In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, pp. 1–7, June 2007
Karcher, H.: Riemannian center of mass and mollifier smoothing. Comm. Pure Appl. Math. 30, 509–541 (1977)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)
Oreifej, O., Liu, Z.: HON4D: histogram of oriented 4D normals for activity recognition from depth sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp. 716–723 (2013)
Ohn-Bar, E., Trivedi, M.M.: Joint angles similarities and HOG2 for action recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2013, Portland, OR, USA, 23–28 June 2013, pp. 465–470 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
De Smedt, Q., Wannous, H., Vandeborre, JP. (2018). 3D Hand Gesture Recognition by Analysing Set-of-Joints Trajectories. In: Wannous, H., Pala, P., Daoudi, M., Flórez-Revuelta, F. (eds) Understanding Human Activities Through 3D Sensors. UHA3DS 2016. Lecture Notes in Computer Science(), vol 10188. Springer, Cham. https://doi.org/10.1007/978-3-319-91863-1_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-91863-1_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91862-4
Online ISBN: 978-3-319-91863-1
eBook Packages: Computer ScienceComputer Science (R0)