skip to main content
10.1145/3574198.3574223acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicbbeConference Proceedingsconference-collections
research-article

An unobtrusive upper-limb activity recognition system based on deep neural network fusion for stroke survivors

Published: 15 March 2023 Publication History

Abstract

Stroke is a cerebrovascular disease that may cause long-term paralysis. Stroke survivors can recover more quickly with personalized treatment, which often requires the identification and evaluation of daily activities. Most of the existing methods for stroke activity recognition use wearable devices to collect motion and/or electrophysiology signals. However, as most survivors are elderly, the wearing process and operation methods are inevitably inconvenient for them. In this paper, we proposed an unobtrusive upper-limb movement recognition system for stroke survivors based on model fusion via combining three deep neural networks. Specifically, we recruited 16 stroke survivors with different impairment levels. Considering fine and dexterous movements of the upper limbs and hands take an important part in our daily life, fine-grained hand activities are more difficult to recognize. We conducted seventeen hand gesture recognition using video data collected by an Azure Kinect sensor. We compared the performance of three state-of-the-art deep neural networks, namely TSN, I3D, and Slowfast. Moreover, we fused the three models using soft voting. The top1 mean accuracy of our fusion model is 93.45% on our dataset. With our method, it is expected to assist rehabilitation physicians, to formulate the corresponding treatment plan, and make better-personalized treatment for stroke survivors.

References

[1]
J. Mackay, G. A. Mensah, and K. Greenlund. The atlas of heart disease and stroke. World Health Organization, 2004.
[2]
S. I. Lee, “Enabling stroke rehabilitation in home and community settings: A wearable sensor-based approach for upper-limb motor training,” IEEE J. Transl. Eng. Health Med., vol. 6, pp. 1–11, 2018.
[3]
D. Rand and J. J. Eng, "Disparity between functional recovery and daily use of the upper and lower extremities during subacute stroke rehabilitation", Neurorehabilitation and neural repair, vol. 26, no. 1, pp. 76-84, 2012
[4]
M. Baran, N. Lehrer, D. Siwiak, Y. Chen, M. Duff, T. Ingalls, and T. Rikakis. Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors. In EMBC, pages 7602-7605, 2011.
[5]
L. Atallah, B. Lo, R. King and G. -Z. Yang, "Sensor Positioning for Activity Recognition Using Wearable Accelerometers," in IEEE Transactions on Biomedical Circuits and Systems, vol. 5, no. 4, pp. 320-329, Aug. 2011.
[6]
Meng L, Zhang A, Chen C, Wang X, Jiang X, Tao L, Fan J, Wu X, Dai C, Zhang Y, Vanrumste B, Tamura T, Chen W. Exploration of Human Activity Recognition Using a Single Sensor for Stroke Survivors and Able-Bodied People. Sensors. 2021; 21(3):799.
[7]
M. Ma, B. J. Meyer, L. Lin, R. Proffitt, and M. Skubic, "VicoVR-Based Wireless Daily Activity Recognition and Assessment System for Stroke Rehabilitation," 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2018, pp. 1117-1121.
[8]
K. Kardaş, "Video Activity Recognition for Surveillance Systems," 2020 28th Signal Processing and Communications Applications Conference (SIU), 2020, pp. 1-4.
[9]
E. D. O. Simbana, P. S.-H. Baeza, A. J. Huete, and C. Balaguer,“Review of automated systems for upper limbs functional assessment in neurorehabilitation,” IEEE Access, vol. 7, pp. 32352–32367, 2019.
[10]
N. Archana and K. Hareesh, "Real-time Human Activity Recognition Using ResNet and 3D Convolutional Neural Networks," 2021 2nd International Conference on Advances in Computing, Communication, Embedded and Secure Systems (ACCESS), 2021, pp. 173-177.
[11]
L Wang, Y Xiong and W. Zhe, "Temporal Segment Networks: Towards Good Practices for Deep Action Recognition", Computer Vision-14th European Conference (ECCV), pp. 20-36, 2016.
[12]
J. Carreira and A. Zisserman, "Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4724-4733.
[13]
C. Feichtenhofer, H. Fan, J. Malik and K. He, "SlowFast Networks for Video Recognition," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 6201-6210.

Index Terms

  1. An unobtrusive upper-limb activity recognition system based on deep neural network fusion for stroke survivors

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICBBE '22: Proceedings of the 2022 9th International Conference on Biomedical and Bioinformatics Engineering
    November 2022
    306 pages
    ISBN:9781450397223
    DOI:10.1145/3574198
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 March 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Deep neural network
    2. Hand gesture recognition
    3. Stroke

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • Shanghai Municipal Science and Technology International R&D Collaboration Project
    • National Key R&D Program of China

    Conference

    ICBBE 2022

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 42
      Total Downloads
    • Downloads (Last 12 months)12
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media