Skip to main content
SpringerLink
Log in

Dual layer transfer learning for sEMG-based user-independent gesture recognition

  • Original Article
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

During the last few years, significant attention has been paid to surface electromyographic (sEMG) signal–based gesture recognition. Nevertheless, sEMG signal is sensitive to various user-dependent factors, like skin impedance and muscle strength, which causes the existing gesture recognition models not suitable for new users and huge precision dropping. Therefore, we propose a dual layer transfer learning framework, named dualTL, to realize user-independent gesture recognition based on sEMG signal. DualTL is composed of two layers. The first layer of dualTL leverages the correlations of sEMG signal among different users to label partial gestures with high confidence from new users. Then, according to the consistencies of sEMG signal from the same users, the rest gestures are labeled in the second layer. We compare our method with three universal machine learning methods, seven representative transfer learning methods, and two deep learning–based sEMG gesture recognition methods. Experimental results show that the average recognition accuracy of dualTL is 80.17%. Comparing with SMO, KNN, RF, PCA, TCA, STL, and CWT, the performance improves 24.26% approximately.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Zhang Y, Chen Y, Yu H, Yang X, Lu W, Liu H (2018) Wearing-independent hand gesture recognition method based on emg armband. Personal and Ubiquitous Computing 22(3):511–524

    Article  Google Scholar 

  2. Moseley J B, JR F W, Pink M, Perry J, Tibone J (1992) Jobe Emg analysis of the scapular muscles during a shoulder rehabilitation program. Am J Sports Med 20(2):128–134

    Article  Google Scholar 

  3. Kawamoto H, Lee S, Kanbe S, Sankai Y (2003) Power assist method for hal-3 using emg-based feedback controller. In: 2003 IEEE international conference on systems, man and cybernetics, vol 2. IEEE, pp 1648–1653

  4. Sears H H, Shaperman J (1991) Proportional myoelectric hand control: an evaluation. Am J Phys Med Rehabil 70(1):20–28

    Article  Google Scholar 

  5. Neto A F, Celeste W C, Martins V R, Bastos Filho T F, Sarcinelli Filho M (2006) Human-machine interface based on electro-biological signals for mobile vehicles. In: 2006 IEEE International Symposium on Industrial Electronics, vol 4. IEEE, pp 2954–2959

  6. Gao N, Zhao L (2016) A pedestrian dead reckoning system using semg based on activities recognition. In: 2016 IEEE Chinese guidance, navigation and control conference (CGNCC). IEEE, pp 2361–2365

  7. Chen J, Li F, Chen H, Yang S, Wang Y (2019) Dynamic gesture recognition using wireless signals with less disturbance. Pers Ubiquit Comput 23(1):17–27

    Article  Google Scholar 

  8. Song J, Sörös G, Pece F, Hilliges O (2015) Real-time hand gesture recognition on unmodified wearable devices. In: IEEE conference on computer vision and pattern recognition

  9. Ducloux J, Colla P, Petrashin P, Lancioni W, Toledo L (2014) Accelerometer-based hand gesture recognition system for interaction in digital tv. In: 2014 IEEE international instrumentation and measurement technology conference (I2MTC) Proceedings. IEEE, pp 1537–1542

  10. Nandakumar R, Iyer V, Tan D, Gollakota S (2016) Fingerio: using active sonar for fine-grained finger tracking. In: Proceedings of the 2016 CHI conference on human factors in computing systems. ACM, pp 1515–1525

  11. Lien J, Gillian N, Karagozler M E, Amihood P, Schwesig C, Olson E, Raja H, Poupyrev I (2016) Soli: ubiquitous gesture sensing with millimeter wave radar. ACM Trans Graph (TOG) 35(4):142

    Article  Google Scholar 

  12. McIntosh J, Marzo A, Fraser M, Phillips C (2017) Echoflex: hand gesture recognition using ultrasound imaging. In: Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, pp 1923–1934

  13. Zhang X, Chen X, Li Y, Lantz V, Wang K, Yang J (2011) A framework for hand gesture recognition based on accelerometer and emg sensors. IEEE Trans Syst Man Cybern-Part A Syst Humans 41(6):1064–1076

    Article  Google Scholar 

  14. Zhang X, Chen X, Wang W-H, Yang J-H, Lantz V, Wang K-Q (2009) Hand gesture recognition and virtual game control based on 3d accelerometer and emg sensors. In: Proceedings of the 14th international conference on intelligent user interfaces. ACM, pp 401–406

  15. McIntosh J, McNeill C, Fraser M, Kerber F, Löchtefeld M, Krüger A (2016) Empress: practical hand gesture classification with wrist-mounted emg and pressure sensing. In: Proceedings of the 2016 CHI conference on human factors in computing systems. ACM, pp 2332–2342

  16. Benatti S, Casamassima F, Milosevic B, Farella E, Schönle P, Fateh S, Burger T, Huang Q, Benini L (2015) A versatile embedded platform for emg acquisition and gesture recognition. IEEE Trans biomed circ Syst 9(5):620–630

    Article  Google Scholar 

  17. Matsubara T, Hyon S-H, Morimoto J (2011) Learning and adaptation of a stylistic myoelectric interface: Emg-based robotic control with individual user differences. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), 2011. IEEE, pp 390–395

  18. Merletti R, Parker PA, Parker PJ (2004) Electromyography: physiology, engineering, and non-invasive applications, vol 11. Wiley, New York

    Book  Google Scholar 

  19. Khushaba R N, Al-Ani A, Al-Jumaily A (2010) Orthogonal fuzzy neighborhood discriminant analysis for multifunction myoelectric hand control. IEEE Trans Biomed Eng 57(6):1410

    Article  Google Scholar 

  20. Amma C, Krings T, Böer J, Schultz T (2015) Advancing muscle-computer interfaces with high-density electromyography. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, pp 929–938

  21. David R L, Cristian C L, Humberto L C (2015) Design of an electromyographic mouse. In: 2015 20th symposium on signal processing, images and computer vision (STSIVA). IEEE, pp 1–8

  22. Saponas T S, Tan D S, Morris D, Balakrishnan R, Turner J, Landay J A (2009) Enabling always-available input with muscle-computer interfaces. In: Proceedings of the 22nd annual ACM symposium on user interface software and technology. ACM, pp 167–176

  23. Khushaba R N (2014) Correlation analysis of electromyogram signals for multiuser myoelectric interfaces. IEEE Trans Neural Syst Rehabil Eng 22(4):745–755

    Article  Google Scholar 

  24. Matsubara T, Morimoto J (2013) Bilinear modeling of emg signals to extract user-independent features for multiuser myoelectric interface. IEEE Trans Biomed Eng 60(8):2205–2213

    Article  Google Scholar 

  25. Orabona F, Castellini C, Caputo B, Fiorilla A E, Sandini G (2009) Model adaptation with least-squares svm for adaptive hand prosthetics. In: 2009 ICRA’09. IEEE international conference on robotics and automation. IEEE, pp 2897–2903

  26. Chattopadhyay R, Pradhan G, Panchanathan S (2011) Subject independent computational framework for myoelectric signals. In 2011 IEEE instrumentation and measurement technology conference (I2MTC). IEEE, pp 1–4

  27. Pan S J, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  28. Pan S J, Kwok J T, Yang Q (2008) Transfer learning via dimensionality reduction. AAAI 8:677–682

    Google Scholar 

  29. Marinelli B, Kang M, Martini M, Zech J R, Titano J, Cho S, Costa A B, Oermann E K (2019) Combination of active transfer learning and natural language processing to improve liver volumetry using surrogate metrics with deep learning. Radiology: Artificial Intelligence 1(1):e180019

    Google Scholar 

  30. Yin X, Yu X, Sohn K, Liu X, Chandraker M (2018) Feature transfer learning for deep face recognition with long-tail data. arXiv:1803.09014

  31. Zhao Z, Chen Y, Liu J, Shen Z, Liu M (2011) Cross-people mobile-phone based activity recognition. In 22nd international joint conference on artificial intelligence

  32. Wang Z, Guo B, Yu Z, Zhou X (2018) Wi-fi csi-based behavior recognition: From signals and actions to activities. IEEE Commun Mag 56(5):109–115

    Article  Google Scholar 

  33. Yu Z, Du H, Yi F, Wang Z, Guo B (2019) Ten scientific problems in human behavior understanding. CCF Trans Pervasive Comput Int 1(1):3–9

    Article  Google Scholar 

  34. Goussies N A, Ubalde S, Mejail M (2014) Transfer learning decision forests for gesture recognition. The J Mach Learn Res 15(1):3667–3690

    MathSciNet  MATH  Google Scholar 

  35. Costante G, Galieni V, Yan Y, Fravolini M L, Ricci E, Valigi P (2014) Exploiting transfer learning for personalized view invariant gesture recognition. In: 2014 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1250–1254

  36. Ozcan T, Basturk A (2019) Transfer learning-based convolutional neural networks with heuristic optimization for hand gesture recognition. Neural Comput Appl 31(12):8955–8970

    Article  Google Scholar 

  37. Cote-Allard U, Fall C L, Campeau-Lecours A, Gosselin C, Laviolette F, Gosselin B (2017) Transfer learning for semg hand gestures recognition using convolutional neural networks. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 1663–1668

  38. Bu Q, Yang G, Feng J, Ming X (2018) Wi-fi based gesture recognition using deep transfer learning. In: 2018 IEEE SmartWorld, ubiquitous intelligence & computing, advanced & trusted computing, scalable computing & communications, cloud & big data computing, internet of people and smart city innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). IEEE, pp 590–595

  39. Oskoei M A, Hu H (2007) Myoelectric control systems—a survey. Biomed Signal Process Control 2 (4):275–294

    Article  Google Scholar 

  40. Rechy-Ramirez E J, Hu H (2011) Stages for developing control systems using emg and eeg signals: a survey, School of computer science and electronic engineering, University of Essex, pp 1744–8050

  41. Phinyomark A, Limsakul C, Phukpattaranont P (2009) A novel feature extraction for robust emg pattern recognition. arXiv:0912.3973

  42. Zeng Z-Q, Yu H-B, Xu H-R, Xie Y-Q, Gao J (2008) Fast training support vector machines using parallel sequential minimal optimization. In: 2008 3rd international conference on intelligent system and knowledge engineering, vol 1. IEEE, pp 997–1001

  43. Cover T M, Hart P E, et al. (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13 (1):21–27

    Article  Google Scholar 

  44. Liaw A, Wiener M, et al. (2002) Classification and regression by randomforest. R news 2(3):18–22

    Google Scholar 

  45. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemometr Intell Laboratory Syst 2(1-3):37–52

    Article  Google Scholar 

  46. Pan S J, Tsang I W, Kwok J T, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  47. Long M, Wang J, Ding G, Sun J, Yu P S (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207

  48. Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE international conference on data mining (ICDM). IEEE, pp 1129–1134

  49. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073

  50. Liu J, Li J, Lu K (2018) Coupled local–global adaptation for multi-source transfer learning. Neurocomputing 275:247–254

    Article  Google Scholar 

  51. Wang J, Chen Y, Hu L, Peng X, Philip S Y (2018) Stratified transfer learning for cross-domain activity recognition. In: 2018 IEEE international conference on pervasive computing and communications (PerCom). IEEE, pp 1–10

  52. Côté-Allard U, Fall CL, Drouin A, Campeau-Lecours A, Gosselin C, Glette K, Laviolette F, Gosselin B (2019) Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans Neural Syst Rehabil Eng 27(4):760–771

    Article  Google Scholar 

Download references

Funding

This work is financially supported by the National Key Research and Development Plan of China (2017YFB1002801); Natural Science Foundation of China under Grant No. 61502456 and No. 61972383; R & D Plan in Key Field of Guangdong Province (No. 2019B010109001); and by Alibaba Group through Alibaba Innovative Research (AIR) Program.

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academic of Sciences, Beijing, China

    Yingwei Zhang, Yiqiang Chen, Hanchao Yu, Xiaodong Yang & Wang Lu

  2. University of Chinese Academy of Sciences, Beijing, China

    Yingwei Zhang, Yiqiang Chen, Xiaodong Yang & Wang Lu

  3. Pengcheng Laboratory, Shenzhen, China

    Yiqiang Chen

Authors
  1. Yingwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiqiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hanchao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaodong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yiqiang Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Chen, Y., Yu, H. et al. Dual layer transfer learning for sEMG-based user-independent gesture recognition. Pers Ubiquit Comput 26, 575–586 (2022). https://doi.org/10.1007/s00779-020-01397-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-020-01397-0

Keywords

Search

Navigation