Skip to main content
Log in

Gesture recognition based on Gramian angular difference field and multi-stream fusion methods

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Surface electromyography-based gesture recognition was widely applied in human–computer interaction, hand rehabilitation, prosthetic control, and other fields. Electromyography (EMG) signals-based gesture classification usually relies on handcrafted feature extraction with intense subjectivity or convolutional neural networks with redundant structures to extract features. This paper converts the raw EMG signals into Gramian Angular Difference Field (GADF) and Gramian Angular Summation Field images. Four models were used to classify the pictures: K-nearest Neighbors (KNN), Generalized Learning Systems, Binary Trees, and Convolutional Neural Networks using MobileNetv1, and the proposed method was verified by using the public dataset NinaproDB2. Experimental results: When the window size is 300 ms, the step size is 10 ms, and KNN are used as the classification model, the average accuracy of EMG signals classification based on the GADF method is 98.17%, and the accuracy of exercises B, C and D was 96.65%, 95.53%, and 98.02%, respectively. The recognition accuracy was 7.92%, 14.25%, and 4.279% higher than the provided baseline.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

No datasets were generated or analysed during the current study.

References

  1. Wang, Y., Wu, Q., et al.: Deep back propagation-long short-term memory network based upper-limb sEMG signal classification for automated rehabilitation. Biomed. Signal Process. Control 40(3), 987–1001 (2020)

    MATH  Google Scholar 

  2. Bi, L., Feleke, A.G., et al.: A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration. Biomed. Signal Process. Control 51, 113–127 (2019)

    Article  MATH  Google Scholar 

  3. Arteaga, M.V., Castiblanco, J.C., et al.: EMG-driven hand model based on the classification of individual finger movements. Biomed. Signal Process. Control 58, 101834 (2020)

    Article  MATH  Google Scholar 

  4. Zhang, L., Long, J., et al.: Estimation of the continuous pronation—supination movement by using multichannel EMG signal features and Kalman filter: application to control an exoskeleton. Front. Bioeng. Biotechnol. 9, 771255 (2022)

    Article  Google Scholar 

  5. Zou, Y., Cheng, L., et al.: Reconstruction of incomplete surface electromyography based on an adversarial autoencoder network. Biomed. Signal Process. Control 86, 105084 (2023)

    Article  MATH  Google Scholar 

  6. Karnam, N.K., Turlapaty, A.C., et al.: Classification of sEMG signals of hand gestures based on energy features. Biomed. Signal Process. Control 70, 102948 (2021)

    Article  MATH  Google Scholar 

  7. Zhang, L., Shi, Y., et al.: Real-time and user-independent feature classification of forearm using EMG signals. J. Soc. Inform. Display 27(2), 101–107 (2019)

    Article  MATH  Google Scholar 

  8. Wang, N., Lao, K., et al.: The recognition of grasping force using LDA. Biomed. Signal Process. Control 47, 393–400 (2019)

    Article  MATH  Google Scholar 

  9. Gao, Z., Wang, Y., et al.: A multifeatured time–frequency neural network system for classifying sEMG. IEEE Trans. Circuits Syst. II Express Briefs 69(11), 4588–4592 (2022)

    MATH  Google Scholar 

  10. Hazarika, A., Barman, P., et al.: Real-time implementation of a multidomain feature fusion model using inherently available large sensor data. IEEE Trans. Ind. Inf. 15(12), 6231–6239 (2019)

    Article  MATH  Google Scholar 

  11. Xu, M., Chen, X., et al.: Cross-user electromyography pattern recognition based on a novel spatial-temporal graph convolutional network. IEEE Trans. Neural Syst. Rehab. Eng. 32, 72–82 (2024)

    Article  MATH  Google Scholar 

  12. Fajardo, J.M., Gomez, O., et al.: EMG hand gesture classification using handcrafted and deep features. Biomed. Signal Process. Control 63, 102210 (2021)

    Article  MATH  Google Scholar 

  13. Azhiri, R.B., Esmaeili, M., et al.: Raw EMG classification using extreme value machine. Biomed. Signal Process. Control 86, 105345 (2023)

    Article  Google Scholar 

  14. Wei, Y., Zhou, J., et al.: A review of algorithm & hardware design for ai-based biomedical applications. IEEE Trans. Biomed. Circuits Syst. 14(2), 145–163 (2020)

    Article  MATH  Google Scholar 

  15. Wang, W., Zhang, J., et al.: Motion intensity modeling and trajectory control of upper limb rehabilitation exoskeleton robot based on multi-modal information. Complex Intell. Syst. 8(3), 2091–2103 (2022)

    Article  MATH  Google Scholar 

  16. Côté-Allard, U., Fall, C.L., et al.: Deep learning for electromyographic hand gesture signal classification using transfer learning. IEEE Trans. Neural Syst. Rehabil. Eng. 27(4), 760–771 (2019)

    Article  MATH  Google Scholar 

  17. Ozdemir, M.A., Kisa, D.H., et al.: Hand gesture classification using time–frequency images and transfer learning based on CNN. Biomed. Signal Process. Control 77, 103787 (2022)

    Article  MATH  Google Scholar 

  18. Karheily, S., Moukadem, A., et al.: SEMG time-frequency features for hand movements classification Expert Syst. Appl 210, 118282 (2022)

    Google Scholar 

  19. Xu, L., Zhang, K., et al.: Gesture recognition using dual-stream CNN based on fusion of sEMG energy kernel phase portrait and imu amplitude image. Biomed. Signal Process. Control 73, 103364 (2022)

    Article  MATH  Google Scholar 

  20. Wu, Y., Liang, S., et al.: Prediction and classification of sEMG-based pinch force between different fingers. Expert Syst. Appl. 237, 121635 (2024)

    Article  Google Scholar 

  21. Qi, P., Xu, H., et al.: Residual neural networks based on empirical mode decomposition for mitral regurgitation prediction. Biomed. Signal Process. Control 86, 105265 (2023)

    Article  MATH  Google Scholar 

  22. Wang, W., Zhao, C., et al.: Research on multimodal fusion recognition method of upper limb motion patterns. IEEE Trans. Instrum. Meas. 72, 1–12 (2023)

    MATH  Google Scholar 

  23. Fu, R., Zhang, B., et al.: Gesture recognition of sEMG signal based on GASF-LDA feature enhancement and adaptive ABC optimized SVM. Biomed. Signal Process. Control 85, 105104 (2023)

    Article  MATH  Google Scholar 

  24. Atzori, M., Gijsberts, A., et al.: Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 1, 1–13 (2014)

    Article  MATH  Google Scholar 

  25. Sahu, P., Singh, B.K., et al.: An improved feature selection approach using global best guided gaussian artificial bee colony for EMG classification. Biomed. Signal Process. Control 80, 104399 (2023)

    Article  MATH  Google Scholar 

  26. Torres-Castilllo, J.R., Lopez-Lopez, C.O., et al.: Neuromuscular disorders detection through time-frequency analysis and classification of multi-muscular EMG signals using Hilbert-Huang transform. Biomed. Signal Process. Control 71, 103037 (2022)

    Article  MATH  Google Scholar 

  27. Zhang, L., Bian, H., et al.: Subjects self-select forearm gestures identification based on post-processing and integrating sEMG sensors and as. Biomed. Signal Process. Control 85, 105023 (2023)

    Article  MATH  Google Scholar 

  28. Qin, X., Yang, D., et al.: Research on emotion recognition of bimodal bioelectrical features based on d-s evidence theory. Biomed. Signal Process. Control 87, 105538 (2024)

    Article  MATH  Google Scholar 

  29. Zhang, Y., Xia, C., et al.: Pattern recognition of hand movements based on multi-channel mechanomyography in the condition of one-time collection and sensor doffing and donning. Biomed. Signal Process. Control 91, 106078 (2024)

    Article  Google Scholar 

  30. Tang, B., Li, R., et al.: A membership-function-based broad learning system for human–robot interaction force estimation under drawing task. Med. Biol. Eng. Compu. 61(8), 1975–1992 (2023)

    Article  MathSciNet  MATH  Google Scholar 

  31. Zhong, J., Li, J., et al.: An incremental cross-modal transfer learning method for gesture interaction. Robot. Auton. Syst. 155, 104181 (2022)

    Article  MATH  Google Scholar 

  32. Li, G., Zou, C., Jiang, G., et al.: Multi-view fusion network-based gesture recognition using sEMG data. IEEE J. Biomed. Health Inf. 28(8), 4432–4443 (2024)

    Article  MATH  Google Scholar 

  33. Bu, D., Guo, S., et al.: SEMG-based motion recognition of upper limb rehabilitation using the improved yolo-v4 algorithm. Life-Basel 12(1), 64 (2022)

    Article  MATH  Google Scholar 

  34. Fu, Y., Song, W., et al.: Feature recognition in multiple CNNs using sEMG images from a prototype comfort test. Comput. Methods Programs Biomed. 243, 107897 (2024)

    Article  MATH  Google Scholar 

  35. Nan, Y., Ju, J., et al.: A-mobilenet : an approach of facial expression recognition. Alex. Eng. J. 61(6), 4435–4444 (2022)

    Article  MATH  Google Scholar 

  36. Xiao, F., Chen, Y., et al.: GADF/GASF-HOG: feature extraction methods for hand movement classification from surface electromyography. J. Neural Eng. 17(4), 046016 (2020)

    Article  Google Scholar 

  37. Sun, B., Song, B., et al.: A multiscale feature extraction network based on channel-spatial attention for electromyographic signal classification. IEEE Trans. Cogn. Dev. Syst. 15(2), 591–601 (2023)

    Article  MATH  Google Scholar 

  38. Zabihi, S., Rahimian, E., et al.: TraHGR: transformer for hand gesture recognition via electromyography. IEEE Trans. Neural Syst. Rehabil. Eng. 31, 4211–4224 (2023)

    Article  MATH  Google Scholar 

Download references

Funding

Xi'an Beilin District Science and Technology Plan Project (GX2307). Colleges and universities scientific and technological personnel service enterprise project (2024JH-GXFW-0019).

Author information

Authors and Affiliations

Authors

Contributions

Huarui Bian: Conceptualization, Writing – original draft, Software. Lei Zhang: Data curation, Methodology, Writing – review & editing.

Corresponding author

Correspondence to Lei Zhang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval

This study involving human participants has been reviewed and approved by the Medical and Experimental Animal Ethics Committee of Northwestern Polytechnical University (202002020). In accordance with national legislation and institutional requirements, written informed consent from participants was not required for this study.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent to publish

The authors affirm that human research participants provided informed consent for publication of the images.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bian, H., Zhang, L. Gesture recognition based on Gramian angular difference field and multi-stream fusion methods. SIViP 19, 120 (2025). https://doi.org/10.1007/s11760-024-03565-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03565-8

Keywords

Navigation