Skip to main content

Advertisement

Springer Nature Link
Log in

Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Bio-signal based hand motion recognition plays a critical role in the tasks of human-machine interaction, such as the natural control of multifunctional prostheses. Although a large number of classification technologies have been taken to improve the motion recognition accuracy, it is still a challenge to achieve acceptable performance for multiple modality input. This study proposes a multi-modality deep forest (MMDF) framework to identify hand motions, in which surface electromyographic signals (sEMG) and acceleration signals (ACC) are fused at the input level. The proposed MMDF framework constitutes of three main stages, sEMG and ACC feature extraction, feature dimension reduction, and a cascade structure deep forest for classification. A public database “Ninapro DB7” is used to evaluate the performance of the proposed framework, and the experimental results show that it can achieve a significantly higher accuracy than that of competitors. Besides, our experimental results also show that MMDF outperforms other traditional classifiers with the input of the single modality of sEMG signals. In sum, this study verifies that ACC signals can be an excellent supplementary for sEMG, and MMDF is a plausible solution to fuse mulit-modality bio-signals for human motion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Joshi D, Nakamura BH, Hahn ME (2015) High energy spectrogram with integrated prior knowledge for EMG-based locomotion classification. Med Eng Phys 37(5):518–524

    Article  Google Scholar 

  2. Matrone GC, Cipriani C, Secco EL, Magenes G, Carrozza MC (2010) Principal components analysis based control of a multi-DoF underactuated prosthetic hand. J Neuroeng Rehabil 7(1):1–13

    Article  Google Scholar 

  3. Kuzborskij I, Gijsberts A, Caputo B.(2012) On the challenge of classifying 52 hand movements from surface electromyography. In: 2012 Annual international conference of the IEEE engineering in medicine and biology society, IEEE, p 4931–4937

  4. Liu J, Chen W, Li M, Kang X (2016) Continuous recognition of multifunctional finger and wrist movements in amputee subjects based on sEMG and accelerometry. Open Biomed Eng J 10:101

    Article  Google Scholar 

  5. Chen H, Zhang Y, Li G, Fang Y, Liu H (2020) Surface electromyography feature extraction via convolutional neural network. Int J Mach Learn Cybern 11(1):185–196

    Article  Google Scholar 

  6. Nogales RE, Benalcázar ME (2021) Hand gesture recognition using machine learning and infrared information: a systematic literature review. Int J Mach Learn Cybern 12(10):2859–2886

    Article  Google Scholar 

  7. Atzori M, Gijsberts A, Castellini C, Caputo B, Hager A-GM, Elsig S, Giatsidis G, Bassetto F, Müller H (2014) Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci Data 1(1):1–13

    Article  Google Scholar 

  8. Atzori M, Gijsberts A, Kuzborskij I, Elsig S, Hager A-GM, Deriaz O, Castellini C, Müller H, Caputo B (2014) Characterization of a benchmark database for myoelectric movement classification. IEEE Trans Neural Syst Rehabil Eng 23(1):73–83

    Article  Google Scholar 

  9. Gijsberts A, Atzori M, Castellini C, Müller H, Caputo B (2014) Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification. IEEE Trans Neural Syst Rehabil Eng 22(4):735–744

    Article  Google Scholar 

  10. Zhou Z-H, Feng J (2019) deep forest: towards an alternative to deep neural networks. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence (IJCAI-17), p 3553–3559

  11. Jiang N, Muceli S, Graimann B, Farina D (2013) Effect of arm position on the prediction of kinematics from EMG in amputees. Med Biol Eng Cmput 51(1):143–151

    Article  Google Scholar 

  12. Fougner A, Scheme E, Chan AD, Englehart K, Stavdahl Ø (2011) Resolving the limb position effect in myoelectric pattern recognition. IEEE Trans Neural Syst Rehabil Eng 19(6):644–651

    Article  Google Scholar 

  13. Geng Y, Zhou P, Li G (2012) Toward attenuating the impact of arm positions on electromyography pattern-recognition based motion classification in transradial amputees. J Neuroeng Rehabil 9(1):1–11

    Article  Google Scholar 

  14. Liu L, Chen X, Lu Z, Cao S, Wu D, Zhang X (2016) Development of an EMG-ACC-based upper limb rehabilitation training system. IEEE Trans Neural Syst Rehabil Eng 25(3):244–253

    Article  Google Scholar 

  15. Zhang X, Chen X, Li Y, Lantz V, Wang K, Yang J (2011) A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans Syst Man Cybern Part A Syst Hum 41(6):1064–1076

    Article  Google Scholar 

  16. Khushaba RN, Al-Timemy A, Kodagoda S, Nazarpour K (2016) Combined influence of forearm orientation and muscular contraction on EMG pattern recognition. Expert Syst Appl 61:154–161

    Article  Google Scholar 

  17. Young A, Kuiken T, Hargrove L (2014) Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses. J Neural Eng 11(5):056021

    Article  Google Scholar 

  18. Atrey PK, Hossain MA, El Saddik A, Kankanhalli MS (2010) Multimodal fusion for multimedia analysis: a survey. Multimedia Syst 16(6):345–379

    Article  Google Scholar 

  19. Xie X, Liu Z (2017) Dynamic gesture recognition method based on EMG and ACC signal. J Comput Appl 37:2700

    Google Scholar 

  20. Krasoulis A, Kyranou I, Erden MS, Nazarpour K, Vijayakumar S (2017) Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements. J Neuroeng Rehabil 14(1):1–14

    Article  Google Scholar 

  21. Guo Y, Liu S, Li Z, Shang X (2018) Bcdforest: a boosting cascade deep forest model towards the classification of cancer subtypes based on gene expression data. BMC Bioinform 19(5):1–13

    Google Scholar 

  22. Liu P, Wang X, Yin L, Liu B (2020) Flat random forest: a new ensemble learning method towards better training efficiency and adaptive model size to deep forest. Int J Mach Learn Cybern 11:2501–2513

    Article  Google Scholar 

  23. Zhang Y, Xu T, Chen C, Wang G, Zhang Z, Xiao T (2021) A hierarchical method based on improved deep forest and case-based reasoning for railway turnout fault diagnosis. Eng Fail Anal 127:105446

    Article  Google Scholar 

  24. Sun L, Mo Z, Yan F, Xia L, Shan F, Ding Z, Song B, Gao W, Shao W, Shi F (2020) Adaptive feature selection guided deep forest for COVID-19 classification with chest CT. IEEE J Biomed Health Inform 24(10):2798–2805

    Article  Google Scholar 

  25. Daouadi KE, Rebaï RZ, Amous I (2021) Optimizing semantic deep forest for tweet topic classification. Inf Syst 101:10–18

    Article  Google Scholar 

  26. Ding J, Wu Y, Luo Q, Du Y (2021) A fault diagnosis method of mechanical bearing based on the deep forest. J Vib Shock 40:107–113

    Google Scholar 

  27. Fang Y, Yang H, Zhang X, Liu H, Tao B (2020) Multi-feature input deep forest for EEG-based emotion recognition. Front Neurorobotics 14:617531

    Article  Google Scholar 

  28. Englehart K, Hudgins B, Parker PA, Stevenson M (1999) Classification of the myoelectric signal using time-frequency based representations. Med Eng Phys 21(6–7):431–438

    Article  Google Scholar 

  29. Oskoei MA, Hu H (2008) Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Trans Biomed Eng 55(8):1956–1965

    Article  Google Scholar 

  30. Fougner A, Scheme E, Chan AD, Englehart K, Stavdahl Ø (2011) A multi-modal approach for hand motion classification using surface emg and accelerometers. In: 2011 Annual international conference of the IEEE engineering in medicine and biology society, IEEE, p 4247–4250

  31. Phinyomark A, Phukpattaranont P, Limsakul C (2012) Feature reduction and selection for EMG signal classification. Expert Syst Appl 39(8):7420–7431

    Article  Google Scholar 

  32. Fang Y, Yang J, Zhou D, Ju Z (2021) Modelling EMG driven wrist movements using a bio-inspired neural network. Neurocomputing 470:89–98

    Article  Google Scholar 

  33. Cheng J, Chen M, Li C, Liu Y, Song R, Liu A, Chen X (2020) Emotion recognition from multi-channel EEG via deep forest. IEEE J Biomed Health Inf 25(2):453–464

    Article  Google Scholar 

  34. Yao H, He H, Wang S, Xie Z 2019) EEG-based emotion recognition using multi-scale window deep forest. In: 2019 IEEE symposium series on computational intelligence (SSCI), IEEE, p 381–386 (

  35. Sebelius FC, Rosen BN, Lundborg GN (2005) Refined myoelectric control in below-elbow amputees using artificial neural networks and a data glove. J Hand Surg 30(4):780–789

    Article  Google Scholar 

  36. Liu FT, Ting KM, Yu Y, Zhou Z-H (2008) Spectrum of variable-random trees. J Artif Intell Res 32:355–384

    Article  MATH  Google Scholar 

  37. Kong D, Zhu J (2019) Gesture recognition based on fusion of surface electromyography and acceleration information. Electron Meas Technol

  38. Zhai Y, Lv P, Deng W, Xie X, Yu C, Gan J, Zeng J, Ying Z, Labati RD, Piuri V (2020) Facial beauty prediction via deep cascaded forest. Int J High Perform Syst Archit 9(2–3):97–106

    Article  Google Scholar 

  39. Liu W, Fan H, Xia M (2021) Step-wise multi-grained augmented gradient boosting decision trees for credit scoring. Eng Appl Artif Intell 97:104036

    Article  Google Scholar 

  40. Fang Y, Hettiarachchi N, Zhou D, Liu H (2015) Multi-modal sensing techniques for interfacing hand prostheses: a review. IEEE Sens J 15(11):6065–6076

    Article  Google Scholar 

  41. Castellini C, Van Der Smagt P (2009) Surface EMG in advanced hand prosthetics. Biol Cybern 100(1):35–47

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the basic public welfare research program of Zhejiang Province (No. LQ20F020016), the national natural science foundation of China (No. 62106147).

Author information

Authors and Affiliations

  1. School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, 310018, Zhejiang, China

    Yinfeng Fang & Huiqiao Lu

  2. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, 518060, Guangdong, China

    Han Liu

Authors
  1. Yinfeng Fang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Huiqiao Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Han Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yinfeng Fang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, Y., Lu, H. & Liu, H. Multi-modality deep forest for hand motion recognition via fusing sEMG and acceleration signals. Int. J. Mach. Learn. & Cyber. 14, 1119–1131 (2023). https://doi.org/10.1007/s13042-022-01687-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01687-4

Keywords