Skip to main content
SpringerLink
Log in

Gesture Recognition Based on Kinect and sEMG Signal Fusion

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

A weighted fusion method of D-S evidence theory in decision making is proposed to aim at the problem of lacking in the distribution of trust, data processing and precision in D-S evidential theory. The method of gesture recognition based on Kinect and sEMG signal are established. Weighted D-S evidence theory is used to fuse Kinect and sEMG signals and the simulation experiment is made respectively. The stimulation results show that comparing with other experimental methods, the decision fusion method based on weighted D-S evidence theory has higher utilization efficiency and recognition rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Guan R, Xu XM, Luo YY (2013) A computer vision-based gesture detection and recognition technique. Computer Applications and Software 30(1):155–159

    Google Scholar 

  2. Li J, Gu D, Liu F (2015) Finger and hand tracking based on kinect depth information. Computer Applications and Software 32(3):79–83

    Google Scholar 

  3. Fang YF, Liu HH, Li GF, Zhu XY (2015) A multichannel surface EMG system for hand motion recognition. International Journal of Humanoid Robotics 12(2):1–11

    Article  Google Scholar 

  4. Liu Y, Yang WL (2016) Hand gesture recognition based on Kinect and Pseudo-Jacobi-Fourier moments. Transducer and Microsystem Technologies 35(7):48–54

    Google Scholar 

  5. Lee HK, Shin SG, Kwon DS (2017) Design of emergency braking algorithm for pedestrian protection based on multi-sensor fusion. Int J Automot Technol 18(6):1067–1076

    Article  Google Scholar 

  6. Belmonte-Hernandez A, Hernandez-Penaloza G, Alvarez F, Conti G (2017) Adaptive Fingerprinting in Multi-Sensor Fusion for Accurate Indoor Tracking. IEEE Sensors J 17(15):4983–4998

    Article  Google Scholar 

  7. Das S, Barani S, Wagh S, Sonavane SS (2017) Extending lifetime of wireless sensor networks using multi-sensor data fusion. Sadhana-Academy Proceedings in Engineering Sciences 42(7):1083–1090

    Google Scholar 

  8. Fu H, Du XK (2005) Multi-sensor optimum fusion based on the bayes estimation. Control Theory and Applications 4(24):10–12

    Google Scholar 

  9. Wadaa A, Olariu S, Wilson L (2005) Training a wireless sensor network. Mobile Networks and Applications 10(1–2):151–168

    Article  Google Scholar 

  10. Yang P, Chen X, Li Y (2012) A sign language recognition method based on multi-sensor information. Space Medicine and Medical Engineering 25(4):276–281

    Google Scholar 

  11. Wang WH, Chen X, Yang P (2010) Chinese sign language recognition based on multiple sensors information detection and fusion. Chin J Biomed Eng 29(5):665–671

    Google Scholar 

  12. Zhao AF, Pei D, Wang QZ (2014) Gesture recognition fused with multi-information in complex environment. Computer Engineering and Applications 50(5):180–184

    Google Scholar 

  13. Chen DS, Li GF, Sun Y, Jiang GZ, Kong JY, Li JH, Liu HH (2017) Fusion hand gesture segmentation and extraction based on CMOS sensor and 3d sensor. Int J Wirel Mob Comput 12(3):305–312

    Article  Google Scholar 

  14. Ju ZJ, Liu HH (2011) A unified fuzzy framework for human-hand motion recognition. IEEE Trans Fuzzy Syst 19(5):901–913

    Article  Google Scholar 

  15. Ju ZJ, Liu HH (2010) Recognizing hand grasp and manipulation through empirical copula. Int J Soc Robot 2(3):321–328

    Article  Google Scholar 

  16. Chen DS, Li GF, Sun Y, Kong JY, Jiang GZ, Tang H, Ju ZJ, Yu H, Liu HH (2017) An interactive image segmentation method in hand gesture recognition. Sensors 17(2):253

    Article  Google Scholar 

  17. Liao YJ, Sun Y, Li GF, Kong JY, Jiang GZ, Jiang D, Cai HB, Ju ZJ, Yu H, Liu HH (2017) Simultaneous calibration: a joint optimization approach for multiple kinect and external cameras. Sensors 17(7):1491

    Article  Google Scholar 

  18. Miao W, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Optimal grasp planning of multi-fingered robotic hands: a review. Applied and Computationnal Mathematics 14(3):238–247

    MathSciNet  MATH  Google Scholar 

  19. Krishna RV, Maziar L, Joohee K (2014) Real-time refinement of kinect depth maps using multi-resolution anisotropic diffusion. Mobile Networks and Applications 19(3):414–425

    Article  Google Scholar 

  20. Li GF, Liu J, Jiang GZ, Liu HH (2015) Numerical simulation of temperature field and thermal stress field in the new type of ladle with the nanometer adiabatic material. Advances in Mechanical Engineering 7(4):1–13

    Google Scholar 

  21. Li GF, Gu YS, Kong JY, Jiang GZ, Xie LX, Wu ZH, Li Z, He Y, Gao P (2013) Intelligent control of air compressor production process. Applied Mathematics & Information Sciences 7(3):1051–1058

    Article  Google Scholar 

  22. Alon J, Athitsos V (2009) A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Transactions on Pattem Analysis and Machine Intelligence 31(9):1685–1699

    Article  Google Scholar 

  23. Li GF, Qu PX, Kong JY, Jiang GZ, Xie LX, Gao P, Wu ZH, He Y (2013) Coke oven intelligent integrated control system. Applied Mathematics & Information Sciences 7(3):1043–1050

    Article  Google Scholar 

  24. Li GF, Miao W, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent control model and its simulation of flue temperature in coke oven. Discrete and Continuous Dynamical Systems - Series S (DCDS-S) 8(6):1223–1237

    Article  MathSciNet  MATH  Google Scholar 

  25. Binh N, Enolkida S, Ejima T (2005). Real-time hand tracking and gesture recognition system. International Conference on Graphic, Vision and Image Processing, pp362–368

  26. Li GF, Kong JY, Jiang GZ, Xie LX, Jiang ZG, Zhao G (2012) Air-fuel ratio intelligent control in coke oven combustion process. Information-An International Interdisciplinary Journal 15(11):4487–4494

    Google Scholar 

  27. Miao W, Li GF, Sun Y, Jiang GZ, Kong JY (2016) Gesture recognition based on sparse representation. Int J Wireless and Mobile Computing 11(4):348–356

    Article  Google Scholar 

  28. Li B, Sun Y, Li GF, Kong JY, Jiang GZ, Jiang D, Tao B, Xu S, Liu HH (2017) Gesture recognition based on modified adaptive orthogonal matching pursuit algorithm. Clust Comput. https://doi.org/10.1007/s10586-017-1231-7

  29. He Y, Li GF, Liao YJ, Sun Y, Kong JY, Jiang GZ, Jiang D, Tao B, Xu S, Liu HH (2017) Gesture recognition based on an improved local sparse representation classification algorithm. Clust Comput. https://doi.org/10.1007/s10586-017-1237-1

  30. Chen Z, Sun SK (2010) A zernike moment phase-based descriptor for local image representation and matching. IEEE Trans Image Process 19(1):205–219

    Article  MathSciNet  MATH  Google Scholar 

  31. Chen DS, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent computational control of multi-fingered dexterous robotic hand. Journal of Computational & Theoretical Nanoscience 12(12):6126–6132

    Article  Google Scholar 

  32. Ding WL, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent computation in grasping control of dexterous robot hand. Journal of Computational & Theoretical Nanoscience 12(12):6096–6099

    Article  Google Scholar 

  33. Liu YQ, Wang XH, Yan K (2016). Hand gesture recognition based on concentric circular scan lines and weighted K-nearest neighbor algorithm. Multimedia Tools and Applications, pp1–15

  34. Hugo JE, Eduardo FM, Enrique SL (2016) A naïve bayes baseline for early gesture recognition. Pattern Recogn Lett 73:91–99

    Article  Google Scholar 

  35. Kim J, Wagner J (2008) Bi-channel sensor fusion for automatic sign language recognition. 8th IEEE International Conference on Automatic Face & Gesture Recognition, pp647–652

  36. Li GF, Liu Z, Jiang GZ, Xiong HG, Liu HH (2015) Numerical simulation of the influence factors for rotary kiln in temperature field and stress field and the structure optimization [J]. Advances in Mechanical Engineering 7(6):1–15

    Article  Google Scholar 

  37. Li GF, Qu PX, Kong JY, Jiang GZ, Xie LX, Wu ZH, Gao P, He Y (2013) Influence of working lining parameters on temperature and stress field of ladle. Applied Mathematics & Information Sciences 7(2):439–448

    Article  Google Scholar 

  38. Vidal C, Jedynak B (2010) Learning to match: deriving optimal template-matching algorithms from probabilistic image models. Int J Comput Vis 88(2):189–213

    Article  MathSciNet  Google Scholar 

  39. Guo WC, Sheng XJ, Liu HG, Zhu XY (2017) Toward an enhanced human-machine interface for upper-limb prosthesis control with combined EMG and NIRS signals. IEEE Transactions on Human-machine Systems 47(4):564–575

    Article  Google Scholar 

  40. Rebecca W, Hussam M, Haydn M, Jiang XQ (2016) Burg algorithm for enhancing measurement performance in wavelength scanning interferometry. Surface Topography: Metrology and Properties 4:1–8

    Google Scholar 

  41. Ding WL, Li GF, Sun Y, Jiang GZ, Kong JY (2017) D-S evidential theory on sEMG signal recognition. Int J Comput Sci Math 8(2):138–145

    Article  MathSciNet  Google Scholar 

  42. Liu S, Fu W, He L et al (2017) Distribution of primary additional errors in fractal encoding method [J]. Multimedia Tools and Applications 76(4):5787–5802

    Article  Google Scholar 

  43. Li Z, Li GF, Jiang GZ, Fang YF, Ju ZJ, Liu HH (2015) Intelligent Computation of grasping and manipulation for multi-fingered robotic hands. Journal of Computational & Theoretical Nanoscience 12(12):6192–6197

    Article  Google Scholar 

  44. Huang Z, Ren FJ (2017) Facial expression recognition based on multi-regional D-S evidences theory fusion. IEEJ Trans Electr Electron Eng 12(2):251–261

    Article  Google Scholar 

  45. Pan Z, Liu S (2017) A review of visual moving target tracking [J]. Multimedia Tools and Applications 76(16):16989–17018

    Article  Google Scholar 

  46. Liu S, Pan Z, Fu W, Cheng X (2017) Fractal generation method based on asymptote family of generalized Mandelbrot set and its application [J]. Journal of Nonlinear Sciences and Applications 10(3):1148–1161

    Article  MathSciNet  Google Scholar 

  47. Li GF, Tang H, Sun Y, Kong JY, Jiang GZ, Jiang D, Tao B, Xu S, Liu HH (2017) Hand gesture recognition based on convolution neural network. Clust Comput. https://doi.org/10.1007/s10586-017-1435-x

Download references

Acknowledgments

This work was supported by grants of National Natural Science Foundation of China (Grant No. 51575407, 51575338, 61273106, 51575412 and 61603420) and the Grants of National Defense Pre-Research Foundation of Wuhan University of Science and Technology (GF201705).

Author information

Authors and Affiliations

  1. Key Laboratory of Metallurgical Equipment and Control Technology of Ministry of Education, Wuhan University of Science and Technology, Wu Han, 430081, China

    Ying Sun, Cuiqiao Li, Gongfa Li & Guozhang Jiang

  2. Hubei Key Laboratory of Mechanical Transmission and Manufacturing Engineering, Wuhan University of Science and Technology, Wuhan, 430081, China

    Ying Sun, Gongfa Li, Guozhang Jiang & Du Jiang

  3. School of Computing, University of Portsmouth, Portsmouth, PO1 3HE, UK

    Honghai Liu

  4. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Zhigao Zheng

  5. College of Computer Science, South-Central University for Nationalities, Wuhan, 430074, China

    Wanneng Shu

Authors
  1. Ying Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cuiqiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gongfa Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guozhang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Du Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Honghai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhigao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wanneng Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Gongfa Li or Zhigao Zheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, Y., Li, C., Li, G. et al. Gesture Recognition Based on Kinect and sEMG Signal Fusion. Mobile Netw Appl 23, 797–805 (2018). https://doi.org/10.1007/s11036-018-1008-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-018-1008-0

Keywords

Search

Navigation