Abstract
In the field of human-computer interaction, vision-based gesture recognition methods are widely studied. However, its recognition effect depends to a large extent on the performance of the recognition algorithm. The skeletonization algorithm and convolutional neural network (CNN) for the recognition algorithm reduce the impact of shooting angle and environment on recognition effect, and improve the accuracy of gesture recognition in complex environments. According to the influence of the shooting angle on the same gesture recognition, the skeletonization algorithm is optimized based on the layer-by-layer stripping concept, so that the key node information in the hand skeleton diagram is extracted. The gesture direction is determined by the spatial coordinate axis of the hand. Based on this, gesture segmentation is implemented to overcome the influence of the environment on the recognition effect. In order to further improve the accuracy of gesture recognition, the ASK gesture database is used to train the convolutional neural network model. The experimental results show that compared with SVM method, dictionary learning + sparse representation, CNN method and other methods, the recognition rate reaches 96.01%.
Similar content being viewed by others
References
Błaszczyk Ł (2016) Compressed sensing in MRI – mathematical preliminaries and basic examples. Nukleonika 61(1):41–43
Bu X, Dong H, Han F et al (2018) Event-triggered distributed filtering over sensor networks with deception attacks and partial measurements. Int J Gen Syst 47:395–407
Caselli N, Sehyr Z, Cohen-Goldberg A et al (2017) ASL-LEX: A lexical database of American Sign Language. Behav Res Methods 49(2):784–801
Chang W, Li G, Kong J et al (2018) Thermal Mechanical Stress Analysis of Ladle Lining with Integral Brick Joint. Arch Metall Mater 63(2):659–666
Chen D, Li G, Sun Y et al (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17(2):253
Fang Y, Liu H, Li G et al (2015) A multichannel surface emg system for hand motion recognition. International Journal of Humanoid Robotics 12(2). https://doi.org/10.1142/S0219843615500115
Han F, Dong H, Wang Z et al (2018) Improved tobit kalman filtering for systems with random parameters via conditional expectation. Signal Process 147. https://doi.org/10.1016/j.sigpro.2018.01.015
He Y, Li G, Liao Y et al (2017) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput 1. https://doi.org/10.1007/s10586-017-1237-1
He Y, Li G, Zhao Y et al (2018) Numerical simulation-based optimization of contact stress distribution and lubrication conditions in the straight worm drive. Strength of Materials 50(11):1–9
Jiang D, Zheng Z, Li G et al (2017) Gesture recognition based on binocular vision. Clust Comput 3. https://doi.org/10.1007/s10586-018-1844-5
Jin KH, Lee D, Ye JC (2017) A general framework for compressed sensing and parallel mri using annihilating filter based low-rank Hankel matrix. IEEE Transactions on Computational Imaging 2(4):480–495
Li G, Gu Y, Kong J et al (2013) Intelligent control of air compressor production process. Applied Mathematics & Information Sciences 7(3):1051–1058
Li G, Kong J, Jiang G et al (2012) Air-fuel ratio intelligent control in coke oven combustion process. International Journal on Information 12(11):4487–4494
Li G, Liu J, Jiang G et al (2015) Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material. Advances in Mechanical Engineering 7(4):1687814015575988
Li G, Liu Z, Jiang G et al (2017) Numerical simulation of the influence factors for rotary kiln in temperature field and stress field and the structure optimization. Advances in Mechanical Engineering 7(6):1687814015589667
Li G, Miao W, Jiang G et al (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete and Continuous Dynamical Systems - Series S 8(6):1223–1237
Li G, Qu P, Kong J et al (2013) Coke oven intelligent integrated control system. Applied Mathematics & Information Sciences 7(3):1043–1050
Li G, Qu P, Kong J et al (2013) Influence of working lining parameters on temperature and stress field of ladle. Applied Mathematics & Information Sciences 7(2):439–448
Li B, Sun Y, Li G et al (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput 1. https://doi.org/10.1007/s10586-017-1231-7
Li G, Tang H, Zhao Y et al (2017) Hand gesture recognition based on convolution neural network. Clust Comput 3. https://doi.org/10.1007/s10586-017-1435-x
Li G, Zhang L, Sun Y et al (2018) Towards the sEMG hand: internet of things sensors and haptic feedback application. Multimed Tools Applications 1:1–18
Liao Y, Sun Y, Li G et al (2017) Simultaneous Calibration: A Joint Optimization Approach for Multiple Kinect and External Cameras. Sensors 17(7):1491. https://doi.org/10.3390/s17071491
Luzanin O, Plancak M (2014) Hand gesture recognition using low-budget data glove and cluster-trained probabilistic neural network. Assem Autom 34(1):94–105
Miao W, Li G, Jiang G et al (2015) Optimal Grasp Planning of Multi-Fingered Robotic Hands: A Review. Applied and computational mathematics 14(3):238–247
Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9):2352–2449
Saha P, Jin D, Liu Y et al (2018) Fuzzy object skeletonization: theory, algorithms, and applications. IEEE Trans Vis Comput Graph 24(8):2298–2314
Shotton J, Fitzgibbon A, Cook M et al (2011) Real-Time Human Pose Recognition in Parts from Single Depth Images. Computer Vision and Pattern Recognition 56(1):1297–1304
Sun Y, Hu J, Li G et al (2018) Gear reducer optimal design based on computer multimedia simulation. J Supercomput 3:1–13. https://doi.org/10.1007/s11227-018-2255-3
Sun Y, Li C, Li G et al (2018) Gesture Recognition Based on Kinect and sEMG Signal Fusion. Mobile Networks and Applications 23(4):797–805. https://doi.org/10.1007/s11036-018-1008-0
Xiong H, Fan H, Jiang G et al (2017) A simulation-based study of dispatching rules in a dynamic job shop scheduling problem with batch release and extended technical precedence constraints. Eur J Oper Res 257(1):13–24
Xiong H, Fan H, Li G et al (2015) Research on steady-state simulation in dynamic job shop scheduling problem. Advances in Mechanical Engineering 7(9):1–11
Yin Q, Li G, Zhu J (2017) Research on the method of step feature extraction for EOD robot based on 2D laser radar. Discrete and Continuous Dynamical Systems - Series S (DCDS-S) 8(6):1415–1421
Zhong X, Chen Y, Yu H et al (2018) Context-Aware Information Based Ultrasonic Gesture Recognition Method. Journal of Computer-Aided Design & Computer Graphics 30(1):173
Cheng W, Sun Y, Li G et al (2018) Jointly network: a network based on CNN and RBM for gesture recognition. Neural Computing and Applications 1–17. https://doi.org/10.1007/s00521-018-3775-8
Acknowledgements
This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 51575412, 61273106, 51505349) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705). This paper is funded by Wuhan University of Science and Technology graduate students short-term study abroad special funds.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Jiang, D., Li, G., Sun, Y. et al. Gesture recognition based on skeletonization algorithm and CNN with ASL database. Multimed Tools Appl 78, 29953–29970 (2019). https://doi.org/10.1007/s11042-018-6748-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-6748-0