Skip to main content

Advertisement

Springer Nature Link
Log in

Improving sEMG-based motion intention recognition for upper-limb amputees using transfer learning

  • S.I: AI-based e-diagnosis
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Hand gesture recognition from multi-channel surface electromyography (sEMG) have been widely studied in the past decade. By analyzing muscle activities measured from forearm muscles, multiple hand gestures can be recognized. This technology can benefit upper-limb amputees in motion intention recognition, especially for those with trans-radial amputation, in terms of prosthesis control, rehabilitation and further human–computer interaction. However, due to the scarcity of signals collected from amputees, many related studies used signals from intact subjects as a proxy and result in overoptimistic classification performance. Comparing to sEMG signals from intact subjects, signals from upper-limb amputees suffer from signal quality deterioration which relates to the level of amputation and maybe other amputation information. Therefore, this study aims at improving the motion intention recognition performance in trans-radial amputated subjects. To tackle the challenges of data scarcity and signal quality deterioration, we propose a CNN-based transfer learning solution leveraging the knowledge learned from sEMG signals of intact subjects. The proposed method was developed from and tested with NinaPro database where 20 intact subjects and 11 amputees. We obtained 67.5% accuracy in the mDWT feature after transfer. And the results improved by 9.4% after transfer compared to no transfer in the RMS feature. In the end of the study, we further discussed the correlation between classification accuracy and amputation information including the percentage of remaining forearm and the number of years since amputation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

source model and target model

Fig. 3

source model. Only use the first three layers weights of the source model

Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Schultz AE, Kuiken TA (2011) Neural interfaces for control of upper limb prostheses: the state of the art and future possibilities. PM&R 3(1):55–67

    Article  Google Scholar 

  2. Oskoei MA, Hu H (2007) Myoelectric control systems—a survey. Biomed Signal Process Control 2(4):275–294

    Article  Google Scholar 

  3. Li X, Chen S, Zhang H, Samuel OW, Wang H, Fang P, Zhang X, Li G (2016) Towards reducing the impacts of unwanted movements on identification of motion intentions. J Electromyogr Kinesiol 28:90–98

    Article  Google Scholar 

  4. Matsubara T, Hyon S-H, and Morimoto J, Learning and adaptation of a stylistic myoelectric interface: Emg-based robotic control with individual user differences. In: 2011 IEEE International Conference on Robotics and Biomimetics, IEEE, 2011, pp. 390–395.

  5. Tommasi T, Orabona F, Castellini C, Caputo B (2012) Improving control of dexterous hand prostheses using adaptive learning. IEEE Trans Rob 29(1):207–219

    Article  Google Scholar 

  6. Kim M, Chung WK, Kim K (2020) Subject-independent semg pattern recognition by using a muscle source activation model. IEEE Robot Automat Lett 5(4):5175–5180

    Article  Google Scholar 

  7. Al-Timemy AH, Khushaba RN, Bugmann G, Escudero J (2015) Improving the performance against force variation of emg controlled multifunctional upper-limb prostheses for transradial amputees. IEEE Trans Neural Syst Rehabil Eng 24(6):650–661

    Article  Google Scholar 

  8. Atzori M, Gijsberts A, Castellini C, Caputo B, Hager A-GM, Elsig S, Giatsidis G, Bassetto F, Muller H (2014) Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Scient Data 1(1):1–13

    Article  Google Scholar 

  9. Gregori V, Gijsberts A, and Caputo B, Adaptive learning to speedup control of prosthetic hands: A few things everybody should know. In: 2017 International Conference on Rehabilitation Robotics (ICORR), IEEE, 2017, pp. 1130–1135.

  10. Zhai X, Jelfs B, Chan RH, Tin C (2017) Self-recalibrating surface emg pattern recognition for neuroprosthesis control based on convolutional neural network. Front Neurosci 11:379

    Article  Google Scholar 

  11. Cene VH, Balbinot A (2019) Enhancing the classification of hand movements through semg signal and non-iterative methods. Heal Technol 9(4):561–577

    Article  Google Scholar 

  12. Lu Z, Chen X, Li Q, Zhang X, Zhou P (2014) A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE Trans Human–Mach Syst 44(2):293–299

    Article  Google Scholar 

  13. Yang G, Deng J, Pang G, Zhang H, Li J, Deng B, Pang Z, Xu J, Jiang M, Liljeberg P et al (2018) An iot-enabled stroke rehabilitation system based on smart wearable armband and machine learning. IEEE J Translat Eng Health Med 6:1–10

    Article  Google Scholar 

  14. Ng CL, Reaz MBI, Chowdhury MEH (2019) A low noise capacitive electromyography monitoring system for remote healthcare applications. IEEE Sens J 20(6):3333–3342

    Article  Google Scholar 

  15. Castellini C, Fiorilla AE, Sandini G (2009) Multi-subject/daily-life activity emg-based control of mechanical hands. J Neuroeng Rehabil 6(1):1–11

    Article  Google Scholar 

  16. Farina D, Cescon C, Merletti R (2002) Influence of anatomical, physical, and detection-system parameters on surface emg. Biol Cybern 86(6):445–456

    Article  MATH  Google Scholar 

  17. Cote-Allard U, Fall CL, Drouin A, Campeau-Lecours A, Gosselin C, Glette K, Laviolette F, Gosselin B (2019) Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans Neural Syst Rehabil Eng 27(4):760–771

    Article  Google Scholar 

  18. Cote-Allard U, Fall CL, Campeau-Lecours A, Gosselin C, Laviolette F, and Gosselin B (2017) Transfer learning for semg hand gestures recognition using convolutional neural networks. In: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, pp. 1663–1668.

  19. Yosinski J, Clune J, Bengio Y, and Lipson H 2014 How transferable are features in deep neural networks?. In: Advances in Neural Information Processing Systems, pp. 3320–3328.

  20. Phinyomark A, Scheme E (2018) EMG pattern recognition in the era of big data and deep learning. Big Data Cognit Comput 2(3):21

    Article  Google Scholar 

  21. Zecca M, Micera S, Carrozza MC, Dario P (2017) Control of multifunctional prosthetic hands by processing the electromyographic signal. Crit Rev Biomed Eng 45(1-6):383-410

    Article  Google Scholar 

  22. Clancy EA, Morin EL, Merletti R (2002) Sampling, noise-reduction and amplitude estimation issues in surface electromyography. J Electromyogr Kinesiol 12(1):1–16

    Article  Google Scholar 

  23. Lucas M-F, Gaufriau A, Pascual S, Doncarli C, Farina D (2008) Multi-channel surface EMG classification using support vector machines and signal-based wavelet optimization. Biomed Signal Process Control 3(2):169–174

    Article  Google Scholar 

  24. Gijsberts A, Atzori M, Castellini C, Müller H, Caputo B (2014) Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification. IEEE Trans Neural Syst Rehabil Eng 22(4):735–744

    Article  Google Scholar 

  25. Cote-Allard U, Nougarou F, Fall CL, Giguere P, Gosselin C, Laviolette F, and Gosselin B, A convolutional neural network for robotic arm guidance using semg based frequency-features. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2016, pp. 2464–2470.

  26. Boschmann A, Platzner M (2014) Towards robust HD EMG pattern recognition: reducing electrode displacement effect using structural similarity. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014, IEEE, PP. 4547-4550

  27. Atzori M, Cognolato M, Muller H (2016) Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front Neurorobot 10:9

    Article  Google Scholar 

  28. Tsinganos P, Cornelis B, Cornelis J, Jansen B, Skodras A (2018) Deep learning in emg-based gesture recognition, In: PhyCS, pp. 107–114

  29. LeCun Y, Jackel L, Bottou L, Brunot A, Cortes C et al (1995) Comparison of learning algorithms for handwritten digit recognition. In: International Conference on Artificial Neural Networks.

  30. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  31. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  32. Pan SJ, Zheng VM, Yang Q, Hu DH (2008) Transfer learning for wifi-based indoor localization. In: Association for the Advancement of Artificial Intelligence (AAAI) workshop, vol. 6, The Association for the Advancement of Artificial Intelligence Palo Alto

  33. Ribani R and Marengoni M, A survey of transfer learning for convolutional neural networks, in: 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI-T), IEEE, 2019, pp. 47–57.

  34. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, and Darrell T, Decaf: A deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, 2014, pp. 647–655.

  35. Du Y, Jin W, Wei W, Hu Y, Geng W (2017) Surface emg-based inter-session gesture recognition enhanced by deep domain adaptation. Sensors 17(3):458

    Article  Google Scholar 

  36. Bengio Y, Deep learning of representations for unsupervised and transfer learning. In: Proceedings of ICML Workshop on Unsupervised and Transfer Learning, 2012, pp. 17–36.

  37. Tosin MC, Cene VH, Balbinot A (2020) Statistical feature and channel selection for upper limb classification using semg signal processing. Res Biomed Eng 36(4):411–427

    Article  Google Scholar 

  38. Sutskever I, Martens J, Dahl G, and Hinton G (2013) On the importance of initialization and momentum in deep learning. In: International Conference on Machine Learning, pp. 1139–1147.

  39. Arunraj M, Srinivasan A, Arjunan SP (2020) A real-time capable linear time classifier scheme for anticipated hand movements recognition from Amputee subjects using surface EMG signals, IRBM

  40. Cene VH, Balbinot A (2020) Resilient EMG classification to enable reliable upper-limb movement intent detection. IEEE Trans Neural Syst Rehabil Eng 28(11):2507–2514

    Article  Google Scholar 

  41. Atzori M, Gijsberts A, Castellini C, Caputo B, Mittaz Hager A-G, Elsig S, Giatsidis G, Bassetto F, Muller H (2016) Effect of clinical parameters on the control of myoelectric robotic prosthetic hands. J Rehabilit Res Develop 53(3):345-358

    Article  Google Scholar 

  42. Wang H, Fang P, Tian L, Zheng Y, Zhou H, Li G, Zhang X, Towards determining the afferent sites of perception feedback on residual arms of amputees with transcutaneous electrical stimulation. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, 2015, pp. 3367–3370

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grants (61,873,349); the General Logistics Department of PLA (BLB19J005); The Guangzhou Science and Technology Planning Project (202,003,000,040).

Author information

Author notes

  1. These authors Jinghua Fan, Mingzhe Jiang contributed equally to this paper.

Authors and Affiliations

  1. School of Biomedical Engineering, Sun Yat-Sen University, Guangzhou, 518056, China

    Jinghua Fan, Mingzhe Jiang & Wanqing Wu

  2. College of Information and Science Technology, Dalian Maritime University, Dalian, China

    Chuang Lin

  3. DACC Laboratory, ZIAT of the Chinese Academy of Science, Zhuhai, China

    Gloria Li

  4. Department of Computer Science, Lakehead University, Thunder Bay, ON, Canada

    Jinan Fiaidhi

  5. College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, China

    Chenfei Ma

Authors
  1. Jinghua Fan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mingzhe Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chuang Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Gloria Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jinan Fiaidhi
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Chenfei Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Wanqing Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wanqing Wu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, J., Jiang, M., Lin, C. et al. Improving sEMG-based motion intention recognition for upper-limb amputees using transfer learning. Neural Comput & Applic 35, 16101–16111 (2023). https://doi.org/10.1007/s00521-021-06292-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06292-0

Keywords