Skip to main content
SpringerLink
Log in

sEMG based hand gesture recognition with deformable convolutional network

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

There is a growing interest in human machine interface and their applications using surface electromyography (sEMG). sEMG based gesture recognition plays a crucial role in interfacing with peripheral devices such as prosthetic hands. Give the challenges in the state of the art of sEMG based gesture recognition using deep learning, we propose a deformable convolutional network (DCN) to optimise the conventional convolution kernels with a goal of achieving better performance of sEMG based gesture recognition. The DCN first apply traditional convolutional layer to obtain low-dimensional feature maps, then use deformable convolutional layer to get high-dimensional feature maps. Moreover, we propose and compare two new image representation methods based on traditional feature extraction, which enable deep learning architectures to extract implicit correlations between different channels from the sparse multichannel sEMG signals. The experiments are conducted to evaluate the proposed methods on three groups of different types and numbers of gestures on the Ninapro-DB1 data set, the proposed DCN has an improvement of 1.1%, 2.6%, and 4.9% compared with traditional CNN, respectively. In addition, the results of experiments indicate that the DCN shows robustness and feasibility in both feature extraction and classification recognition for the sEMG based gesture recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Ahsan MR, Ibrahimy MI, Khalifa OO (2012) Optimization of neural network for efficient EMG signal classification. In: International symposium on mechatronics and its applications

  2. Li G (2011) Electromyography pattern-recognition-based control of powered multifunctional upper-limb prostheses. Adv Appl Electromyogr 6:99–116

    Google Scholar 

  3. Wang R, Huang C, Li B (1996) Discussion on various methods of EMG processing for the control of prostheses. In: Proceedings of international conference on biomedical engineering, pp 341–344

  4. Hu OH (2007) Myoelectric control systems-a survey. Biomed Signal Process Control 2:275–294

    Article  Google Scholar 

  5. Xing K, Yang P, Huang J, Wang Y, Zhu Q (2014) A real-time EMG pattern recognition method for virtual myoelectric hand control. Neurocomputing 136(JUL.20):345–355

    Article  Google Scholar 

  6. Faust O, Hagiwara Y, Hong TJ, Lih OS, Acharya UR (2018) Deep learning for healthcare applications based on physiological signals: a review. Comput Methods Progr Biomed 161:1–13

    Article  Google Scholar 

  7. Rao R, Derakhshani R (2005) A comparison of EEG preprocessing methods using time delay neural networks. In: International IEEE Embs conference on neural engineering

  8. Ron Kohavi A, George B, John H (1997) Wrappers for feature subset selection. Artif Intell 97(1–2):273–324

    Article  Google Scholar 

  9. Lam HK, Ekong U, Xiao B, Ouyang G, Liu H, Chan KY, Ho Ling S (2015) Variable weight neural networks and their applications on material surface and epilepsy seizure phase classifications. Neurocomputing 149:1177–1187

    Article  Google Scholar 

  10. Ekong U, Lam HK, Xiao B, Ouyang G, Hongbin L (2016) Classification of epilepsy seizure phase using interval type-2 fuzzy support vector machines. Neurocomputing 199:66–76

    Article  Google Scholar 

  11. Alty SR, Lam HK, Prada J (2012) On the applications of heart disease risk classification and hand-written character recognition using support vector machines. In: Computational intelligence and its applications: evolutionary computation, Fuzzy Logic, Neural Network and Support Vector Machine Techniques, pp 213–253

  12. Darak BS, Hambarde SM (2015) A review of techniques for extraction of cardiac artifacts in surface EMG signals and results for simulation of ECG–EMG mixture signal. In: 2015 International conference on pervasive computing (ICPC)

  13. Phinyomark A, Quaine F, Charbonnier S, Serviere C, Tarpin-Bernard F, Laurillau Y (2013) EMG feature evaluation for improving myoelectric pattern recognition robustness. Expert Syst Appl 40:4832–4840

    Article  Google Scholar 

  14. Li Y, Zhang Q, Zeng N, Chen J, Zhang Q (2019) Discrete hand motion intention decoding based on transient myoelectric signals. IEEE Access 7:81630–81639

    Article  Google Scholar 

  15. Tsinganos P, Cornelis B, Cornelis J, Jansen B, Skodras A (2018) Deep learning in EMG-based gesture recognition. In: 5th International conference on physiological computing systems

  16. Atzori M, Gijsberts A, Castellini C, Caputo B, Müller H (2014) Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci Data 1:1–13

    Article  Google Scholar 

  17. Pinzón-Arenas JO, Jiménez-Moreno R, Herrera-Benavides JE (2019) Convolutional neural network for hand gesture recognition using 8 different EMG signals. In: Symposium on image, signal processing and artificial vision

  18. Jia G, Lam HK, Liao J, Wang R (2020) Classification of electromyographic hand gesture signals using machine learning techniques. Neurocomputing 401:236–248

    Article  Google Scholar 

  19. Yang W, Yang D, Liu Y, Liu H (2019) Decoding simultaneous multi-DOF wrist movements from raw EMG signals using a convolutional neural network. IEEE Trans Hum Mach Syst 49(5):1–10

    Article  Google Scholar 

  20. Chen X, Li Y, Hu R, Zhang X (2020) Hand gesture recognition based on surface electromyography using convolutional neural network with transfer learning method. IEEE J Biomed Health Inform 25(4):1

    Article  Google Scholar 

  21. Yang W, Yang D, Liu Y, Liu H (2019) Emg pattern recognition using convolutional neural network with different scale signalspectra input. Int J Hum Robot 16(04):305–345

    Article  Google Scholar 

  22. Yongkang Wong, Wentao Wei, Mohan Kankanhalli, Weidong Geng (2018) A novel attention-based hybrid CNN–RNN architecture for SEMG-based gesture recognition. PloS One 13:e0206049

    Article  Google Scholar 

  23. Geng W, Du Y, Jin W, Wei W, Li J (2016) Gesture recognition by instantaneous surface EMG images. Sci Rep 6:36571

    Article  Google Scholar 

  24. Jiang W, Yin Z (2015) Human activity recognition using wearable sensors by deep convolutional neural networks. In: ACM international conference on multimedia, pp 1307–1310

  25. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  26. Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338

    Article  Google Scholar 

  27. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 764–773

  28. Atzori M, Gijsberts A, Heynen S, Mittaz-Hager AG, Mueller H (2012) Building the ninapro database: a resource for the biorobotics community. In: 2012 4th IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics (BioRob)

  29. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  30. Chen T, Li M, Li Y, Lin M, Wang N, Wang M, Xiao T, Xu B, Zhang C, Zhang Z (2015) Mxnet: a flexible and efficient machine learning library for heterogeneous distributed systems. arXiv preprint arXiv:1512.01274

  31. Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: International conference on machine learning, PMLR, pp 1139–1147

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, 310023, China

    Hao Wang & Yue Zhang

  2. Wuhan University of Techonology, Wuhan, 430070, China

    Chao Liu

  3. Harbin Institute of Technology, Shenzhen, 518055, China

    Honghai Liu

Authors
  1. Hao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Honghai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 579 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Zhang, Y., Liu, C. et al. sEMG based hand gesture recognition with deformable convolutional network. Int. J. Mach. Learn. & Cyber. 13, 1729–1738 (2022). https://doi.org/10.1007/s13042-021-01482-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01482-7

Keywords

Search

Navigation