Skip to main content
SpringerLink
Log in
Book cover

International Conference on Physiological Computing Systems

International Conference on Physiological Computing Systems

International Conference on Physiological Computing Systems

PhyCS 2016, PhyCS 2017, PhyCS 2018: Physiological Computing Systems pp 180–197Cite as

Hand Gesture Recognition Based on EMG Data: A Convolutional Neural Network Approach

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10057))

Abstract

Deep learning (DL) has transformed the field of data analysis by dramatically improving the state of the art in various classification and prediction tasks. Especially in the area of computer vision and speech processing, DL has recently demonstrated better performance and generalisation properties, compared to classical machine learning approaches, which are based on the extraction of hand-crafted model-based features followed by classification. Hand gestures and speech constitute two of the most important modalities in human-to-human communication and man-machine interaction. In biomedical engineering, a lot of new work is directed towards electromyography-based gesture recognition. In this paper, we present a brief overview of DL methods for electromyography-based hand gesture recognition and then we select from literature a simple model based on Convolutional Neural Networks that we consider as the baseline model. The proposed modifications to the baseline model yield a 3% classification improvement. In the current paper, we concentrate on the explanatory analysis of this performance improvement. An ablation study identifies which modifications are the most important ones, and label smoothing is investigated to verify if the results can be improved by reducing a priori bias. The analysis helps in understanding the limitations of the model and exploring new ways to improve the performance.

This work was supported by the VUB-UPatras International Joint Research Group (IJRG) on ICT.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Atzori, M., Cognolato, M., Müller, H.: Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front. Neurorobot. 10, 9 (2016). https://doi.org/10.3389/fnbot.2016.00009. http://journal.frontiersin.org/Article/10.3389/fnbot.2016.00009/abstract

    Article  Google Scholar 

  2. Atzori, M., et al.: Electromyography data for non-invasive naturally-controlled robotic hand prostheses. Sci. Data 1, 140053 (2014). https://doi.org/10.1038/sdata.2014.53. http://www.nature.com/articles/sdata201453

    Article  Google Scholar 

  3. Atzori, M., et al.: Characterization of a benchmark database for myoelectric movement classification. IEEE Trans. Neural Syst. Rehabil. Eng. 23(1), 73–83 (2015). https://doi.org/10.1109/TNSRE.2014.2328495. http://ieeexplore.ieee.org/document/6825822/

    Article  Google Scholar 

  4. Bagherinezhad, H., Horton, M., Rastegari, M., Farhadi, A.: Label refinery: improving ImageNet classification through label progression. ArXiv e-prints (May 2018). http://arxiv.org/abs/1805.02641

  5. Castellini, C., Fiorilla, A., Sandini, G.: Multi-subject/daily-life activity EMG-based control of mechanical hands. J. Neuroeng. Rehabil. 6, 41 (2009). https://doi.org/10.1186/1743-0003-6-41. www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC2784470, http://www.ncbi.nlm.nih.gov/pubmed/19919710

  6. Chen, H., Tong, R., Chen, M., Fang, Y., Liu, H.: A hybrid CNN-SVM classifier for hand gesture recognition with surface EMG signals. In: 2018 International Conference on Machine Learning and Cybernetics (ICMLC), pp. 619–624. IEEE, July 2018. https://doi.org/10.1109/ICMLC.2018.8526976, https://ieeexplore.ieee.org/document/8526976/

  7. Cheok, M., Omar, Z., Jaward, M.: A review of hand gesture and sign language recognition techniques. Int. J. Mach. Learn. Cybern. 10, 1–3 (2017). https://doi.org/10.1007/s13042-017-0705-5

    Article  Google Scholar 

  8. Côté-Allard, U., et al.: Deep learning for electromyographic hand gesture signal classification by leveraging transfer learning. ArXiv e-prints (Jan 2018). http://arxiv.org/abs/1801.07756

  9. Ding, Z., Yang, C., Tian, Z., Yi, C., Fu, Y., Jiang, F.: sEMG-based gesture recognition with convolution neural networks. Sustainability 10(6), 1865 (2018). https://doi.org/10.3390/su10061865. http://www.mdpi.com/2071-1050/10/6/1865

    Article  Google Scholar 

  10. Du, Y., Jin, W., Wei, W., Hu, Y., Geng, W.: Surface EMG-based inter-session gesture recognition enhanced by deep domain adaptation. Sensors 17(3), 458 (2017). https://doi.org/10.3390/s17030458. http://www.mdpi.com/1424-8220/17/3/458

    Article  Google Scholar 

  11. Englehart, K., Hudgins, B.: A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 50(7), 848–854 (2003). https://doi.org/10.1109/TBME.2003.813539. http://ieeexplore.ieee.org/document/1206493/

    Article  Google Scholar 

  12. Englehart, K., Hudgins, B., Parker, P., Stevenson, M.: Classification of the myoelectric signal using time-frequency based representations. Med. Eng. Phys. 21(6), 431–438 (1999). https://doi.org/10.1016/S1350-4533(99)00066-1. http://www.sciencedirect.com/science/article/pii/S1350453399000661

    Article  Google Scholar 

  13. Geng, W., Du, Y., Jin, W., Wei, W., Hu, Y., Li, J.: Gesture recognition by instantaneous surface EMG images. Sci. Rep. 6, 36571 (2016). https://doi.org/10.1038/srep36571

    Article  Google Scholar 

  14. Gijsberts, A., Atzori, M., Castellini, C., Müller, H., Caputo, B.: Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification. IEEE Trans. Neural Syst. Rehabil. Eng. 22(4), 735–744 (2014). https://doi.org/10.1109/TNSRE.2014.2303394

    Article  Google Scholar 

  15. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward Neural Networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS). Sardinia, Italy (2010). http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf

  16. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016). www.deeplearningbook.org

    MATH  Google Scholar 

  17. Hartwell, A., Kadirkamanathan, V., Anderson, S.R.: Compact deep neural networks for computationally efficient gesture classification from electromyography signals. In: 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob), pp. 891–896. IEEE, August 2018. https://doi.org/10.1109/BIOROB.2018.8487853, https://ieeexplore.ieee.org/document/8487853/

  18. Hudgins, B., Parker, P., Scott, R.: A new strategy for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 40(1), 82–94 (1993). https://doi.org/10.1109/10.204774. http://ieeexplore.ieee.org/document/204774/

    Article  Google Scholar 

  19. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)0.5MB model size. ArXiv e-prints (Feb 2016). http://arxiv.org/abs/1602.07360

  20. Kuzborskij, I., Gijsberts, A., Caputo, B.: On the challenge of classifying 52 hand movements from surface electromyography. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4931–4937. IEEE, August 2012. https://doi.org/10.1109/EMBC.2012.6347099, http://ieeexplore.ieee.org/document/6347099/

  21. Lea, C., Flynn, M., Vidal, R., Reiter, A., Hager, G.: Temporal convolutional networks for action segmentation and detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1003–1012. IEEE, July 2017. https://doi.org/10.1109/CVPR.2017.113, http://ieeexplore.ieee.org/document/8099596/

  22. Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting Batch Normalization for practical domain adaptation. ArXiv e-prints (Mar 2016). https://arxiv.org/abs/1603.04779

  23. Park, K., Lee, S.: Movement intention decoding based on deep learning for multiuser myoelectric interfaces. In: 2016 4th International Winter Conference on Brain-Computer Interface (BCI), pp. 1–2. IEEE, Febuary 2016. https://doi.org/10.1109/IWW-BCI.2016.7457459, http://ieeexplore.ieee.org/document/7457459/

  24. Pereyra, G., Tucker, G., Chorowski, J., Kaiser, Ł., Hinton, G.: Regularizing Neural Networks by penalizing confident output distributions. ArXiv e-prints (Jan 2017). http://arxiv.org/abs/1701.06548

  25. Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems. ArXiv e-prints (Dec 2015). http://arxiv.org/abs/1512.08756

  26. Samadani, A.: EMG channel selection for improved hand gesture classification. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4297–4300. IEEE, July 2018. https://doi.org/10.1109/EMBC.2018.8513395, https://ieeexplore.ieee.org/document/8513395/

  27. Samadani, A.: Gated recurrent neural networks for EMG-based hand gesture classification. a comparative study. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 1–4. IEEE, July 2018. https://doi.org/10.1109/EMBC.2018.8512531, https://ieeexplore.ieee.org/document/8512531/

  28. Scheme, E., Englehart, K.: Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. J. Rehabil. Res. Dev. 48(6), 643–659 (2011). https://doi.org/10.1682/JRRD.2010.09.0177. http://www.rehab.research.va.gov/jour/11/486/pdf/scheme486.pdf

    Article  Google Scholar 

  29. Sergey, I., Szegedy, C.: Batch Normalization: Accelerating Deep Network training by reducing internal covariate shift. ArXiv e-prints (Feb 2015). https://arxiv.org/abs/1502.03167

  30. Shim, H., An, H., Lee, S., Lee, E., Min, H., Lee, S.: EMG pattern classification by split and merge deep belief network. Symmetry 8(12), 148 (2016). https://doi.org/10.3390/sym8120148. http://www.mdpi.com/2073-8994/8/12/148

    Article  MathSciNet  Google Scholar 

  31. Shim, H., Lee, S.: Multi-channel electromyography pattern classification using deep belief networks for enhanced user experience. J. Cent. South Univ. 22(5), 1801–1808 (2015). https://doi.org/10.1007/s11771-015-2698-0. http://link.springer.com/10.1007/s11771-015-2698-0

    Article  Google Scholar 

  32. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  33. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826. IEEE, June 2016. https://doi.org/10.1109/CVPR.2016.308, http://ieeexplore.ieee.org/document/7780677/

  34. Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., Skodras, A.: Deep learning in EMG-based gesture recognition. In: Proceedings of the 5th International Conference on Physiological Computing Systems, pp. 107–114. SCITEPRESS - Science and Technology Publications, Seville, Spain (2018). https://doi.org/10.5220/0006960201070114, http://www.scitepress.org/DigitalLibrary/Link.aspx?doi=10.5220/0006960201070114

  35. Wei, W., Wong, Y., Du, Y., Hu, Y., Kankanhalli, M., Geng, W.: A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recogn. Lett. (2017). https://doi.org/10.1016/j.patrec.2017.12.005

    Article  Google Scholar 

  36. Zhai, X., Jelfs, B., Chan, R., Tin, C.: Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front. Neurosci. 11, 379–390 (2017). https://doi.org/10.3389/fnins.2017.00379. http://journal.frontiersin.org/article/10.3389/fnins.2017.00379/full

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, University of Patras, 26504, Patras, Greece

    Panagiotis Tsinganos & Athanassios Skodras

  2. Department of Electronics and Informatics, Vrije Universiteit Brussel, 1050, Brussels, Belgium

    Panagiotis Tsinganos, Bruno Cornelis, Jan Cornelis & Bart Jansen

  3. imec, Kapeldreef 75, 3001, Leuven, Belgium

    Bart Jansen

Authors
  1. Panagiotis Tsinganos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruno Cornelis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Cornelis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bart Jansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Athanassios Skodras
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panagiotis Tsinganos .

Editor information

Editors and Affiliations

  1. Medical University of Graz, Graz, Austria

    Andreas Holzinger

  2. NASA Langley Research Center, Hampton, VA, USA

    Alan Pope

  3. IT - Instituto de Telecomunicações, Lisbon, Portugal

    Hugo Plácido da Silva

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., Skodras, A. (2019). Hand Gesture Recognition Based on EMG Data: A Convolutional Neural Network Approach. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds) Physiological Computing Systems. PhyCS PhyCS PhyCS 2016 2017 2018. Lecture Notes in Computer Science(), vol 10057. Springer, Cham. https://doi.org/10.1007/978-3-030-27950-9_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27950-9_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27949-3

  • Online ISBN: 978-3-030-27950-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation