Abstract
Existing gesture-recognition methods have poor performance on the mobile terminal, because of their limited computing resources and noisy environments such as Spatio-temporal variations and light variations. In this paper, we first draw the idea of regression in object detection tasks into dynamic gesture trajectory recognition. We regard gesture trajectory recognition as an object detection task and propose a regression-based dynamic hand gesture detector named RD-Hand, which contains a special full connection layer for trajectory regression. RD-Hand only needs to perform classification and location once at the same time, and the target trajectory can be detected by regression. Firstly, we use a network combined with CNN and LSTM to extract the features of the trajectory. Then we use a full connection layer with special meaning to predict the classification and location. Finally, trajectory-based non-maximum suppression is used to eliminate redundant solutions. Experiments show that RD-Hand has high accuracy, good real-time performance without GPU.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhou Y, Jiang G, Lin Y (2016) A novel finger and hand pose estimation technique for real-time hand gesture recognition. Pattern Recogn 49:102–114
Stergiopoulou E, Papamarkos N (2009) Hand gesture recognition using a neural network shape fitting technique. Eng Appl Artif Intell 22(8):1141–1158
Mirehi N, Tahmasbi M, Targhi AT (2019) Hand gesture recognition using topological features. Multimed Tools Appl 78(10):1–26
De Smedt Q, Wannous H, Vandeborre J (2019) Heterogeneous hand gesture recognition using 3D dynamic skeletal data. Comput Vis Image Underst 181:60–72
Beh J, Han D, Ko H (2014) Rule-based trajectory segmentation for modeling hand motion trajectory. Pattern Recogn 47(4):1586–1601
Wang C, Liu Z, Chan S (2015) Superpixel-based hand gesture recognition with Kinect depth camera. IEEE Trans Multimed 17(1):29–39
Ren Z, Yuan J, Meng J, et al. (2013) Robust part-based hand gesture recognition using Kinect sensor. IEEE Trans Multimed 15(5):1110–1120
Lu W, Tong Z, Chu J (2016) Dynamic hand gesture recognition with leap motion controller. IEEE Signal Process Lett 23(9):1188–1192
Jiang X, Xiao ZG, Menon C (2018) Virtual grasps recognition using fusion of Leap Motion and force myography. Virtual Reality 22(4):297–308
Jian C, Li J, Zhang M (2019) LSTM-based dynamic probability continuous hand gesture trajectory recognition. IET Image Process 13(12):2314–2320
Baranwal N, Nandi GC, Singh AK (2017) Real-time gesture-based communication using possibility theory-based hidden Markov model. Comput Intell 33(4):843–862
Premaratne P, et al. (2017) Centroid tracking based dynamic hand gesture recognition using discrete hidden Markov models. Neurocomputing 228:79–83
Sagayam KM, Hemanth DJ (2018) ABC algorithm based optimization of 1-D hidden Markov model for hand gesture recognition applications. Comput Ind 99:313–323
Boukerma H, et al. (2018) The efficiency of the NSHPZ-HMM: theoretical and practical study. Applied Intelligence (Dordrecht, Netherlands) 48(12):4660–4677
Belgacem S, Chatelain C, Paquet T (2017) Gesture sequence recognition with one shot learned CRF/HMM hybrid model. Image Vis Comput 61:12–21
Roh M, Fazli S, Lee S (2016) Selective temporal filtering and its application to hand gesture recognition. Applied Intelligence (Dordrecht, Netherlands) 45(2):255–264
Choi H, Kim T (2017) Directional dynamic time warping for gesture recognition. https://doi.org/10.1145/3145511.3145526
Plouffe G, Cretu A (2016) Static and dynamic hand gesture recognition in depth data using dynamic time warping. IEEE Trans Instrum Meas 65(2):305–316
Tang J, et al. (2018) Structured dynamic time warping for continuous hand trajectory gesture recognition. Pattern Recogn 80:21–31
Wu XY (2019) A hand gesture recognition algorithm based on DC-CNN. Multimedia Tools and Applications
Wang L et al (2016) Temporal segment networks: towards good practices for deep action recognition. https://doi.org/10.1007/978-3-319-46484-8_2
Zhu G, et al. (2017) Multimodal gesture recognition using 3-D convolution and convolutional LSTM. IEEE Access 5:4517–4524
Jian C, Yang M, Zhang M (2019) Mobile terminal trajectory recognition based on improved LSTM model. IET Image Process 13(11):1914–1921
Li C, et al. (2018) Deep Fisher discriminant learning for mobile hand gesture recognition. Pattern Recogn 77:276–288
Redmon J, et al. (2016) You only look once: unified, real-time object detection. https://doi.org/10.1109/CVPR.2016.91
Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. https://doi.org/10.1109/CVPR.2017.690
Redmon J, Farhadi A (2020) Yolov3: an incremental improvement, CoRR abs/1804.02767. Available from: arXiv:1804.02767
Jian C, Xiang X, Zhang M (2019) Mobile terminal gesture recognition based on improved FAST corner detection. IET Image Process 13(6):991–997
Yang C, Han DK, Ko H (2017) Continuous hand gesture recognition based on trajectory shape information. Pattern Recognit Lett 99:39–47
Collobert R, et al. (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537
Ozturk O, et al. (2015) Boosting real-time recognition of hand posture and gesture for virtual mouse operations with segmentation. Applied Intelligence (Dordrecht, Netherlands) 43(4):786–801
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant No. 61672461 and No. 61672463.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jian, C., Liu, X. & Zhang, M. RD-Hand: a real-time regression-based detector for dynamic hand gesture. Appl Intell 52, 417–428 (2022). https://doi.org/10.1007/s10489-021-02380-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02380-9