Skip to main content
SpringerLink
Log in

Hand rehabilitation assessment system using leap motion controller

  • Original Article
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

This paper presents an approach for monitoring exercises of hand rehabilitation for post stroke patients. The developed solution uses a leap motion controller as hand-tracking device and embeds a supervised machine learning. The K-nearest neighbor methodology is adopted for automatically characterizing the physiotherapist or helper hand movement resulting a unique movement pattern that constitutes the basis of the rehabilitation process. In the second stage, an evaluation of the patients rehabilitation exercises results is compared to the movement pattern of the patient and results are presented, saved and statistically analyzed. Physicians and physiotherapists monitor and assess patients’ rehabilitation improvements through a web application, furthermore, offer medical assisted rehabilitation processes through low cost technology, which can be easily exploited at home. Recorded tracked motion data and results can be used for further medical study and evaluating rehabilitation trends according to patient’s rehabilitation practice and improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  • Aggarwal CC, Zhai C (2012) Mining text data. Springer, Berlin

    Book  Google Scholar 

  • Aguilar-Lazcano CA, Rechy-Ramirez EJ, Hu H et al (2019) Interaction modalities used on serious games for upper limb rehabilitation: a systematic review. Games Health J

  • Anderson KR, Woodbury ML, Phillips K et al (2015) Virtual reality video games to promote movement recovery in stroke rehabilitation: a guide for clinicians. Arch Phys Med Rehabil 96(5):973–976

    Article  Google Scholar 

  • Bamrungthai P, Pleehachinda W (2015) Development of a game-based system to support stroke rehabilitation using kinect device. In: Science and Technology (TICST), 2015 International Conference on; IEEE, p 323–326

  • Bhattacharya S, Czejdo B, Perez N (2012) Gesture classification with machine learning using kinectics sensor data. In: Emerging applications of information technology (EAIT), 2012 third international conference on; IEEE, pp 348–351

  • Butt A, Rovini E, Dolciotti C, et al (2017) Leap motion evaluation for assessment of upper limb motor skills in parkinson’s disease. In: Rehabilitation robotics (ICORR), 2017 international conference on; IEEE, pp 116–121

  • Chang E, Zhao X, Cramer SC et al (2016) Home-based hand rehabilitation after chronic stroke: Randomized, controlled single-blind trial comparing the musicglove with a conventional exercise program. J Rehabil Res Dev 53(4):457

    Article  Google Scholar 

  • Cohen MW, Voldman I, Regazzoni D et al (2018) Hand rehabilitation via gesture recognition using leap motion controller. In: 2018 11th International conference on human system interaction (HSI); IEEE, pp 404–410

  • Colgan A (2014) How does the leap motion controller work? Leap Motion Blog, p 9. https://www.leapmotion.com

  • Cronce A, Gerald Fluet P, Patel J (2018) Home-based virtual rehabilitation for upper extremity functional recovery post-stroke. J Altern Med Res 10(1):27–35

    Google Scholar 

  • Deng Z, Zhu X, Cheng D et al (2016) Efficient knn classification algorithm for big data. Neurocomputing 195:143–148

    Article  Google Scholar 

  • Di Tommaso L, Aubry S, Godard J et al (2016) A new human machine interface in neurosurgery: the leap motion (®). technical note regarding a new touchless interface. Neurochirurgie 62(3):178–181

    Article  Google Scholar 

  • D’Orazio T, Marani R, Renó V et al (2016) Recent trends in gesture recognition: how depth data has improved classical approaches. Image Vis Comput 52:56–72

    Article  Google Scholar 

  • Ebert L, Flach P, Thali M et al (2014) Out of touch-a plugin for controlling osirix with gestures using the leap controller. J Forensic Radiol Imaging 2(3):126–128

    Article  Google Scholar 

  • Estepa A, Piriz SS, Albornoz E et al (2016) Development of a kinect-based exergaming system for motor rehabilitation in neurological disorders. J Phys Conf Ser 705:012060

    Article  Google Scholar 

  • Guna J, Jakus G, Pogačnik M et al (2014) An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking. Sensors 14(2):3702–3720

    Article  Google Scholar 

  • Hidalgo JCC, Bykbaev YR, Delgado JDA et al (2018) Serious game to improve fine motor skills using leap motion. In: 2018 Congreso Argentino de Ciencias de la Informática y Desarrollos de Investigación (CACIDI). IEEE, pp 1–5

  • Hondori HM, Khademi M, Dodakian L et al (2013) A spatial augmented reality rehab system for post-stroke hand rehabilitation. MMVR 184:279–285

    Google Scholar 

  • Huang C (2011) Using sas to find the best k for k-nearest-neighbor classification. SAS programming for data mining applications

  • Ibanez R, Soria Á, Teyseyre A et al (2014) Easy gesture recognition for kinect. Adv Eng Softw 76:171–180

    Article  Google Scholar 

  • Ibañez R, Soria A, Teyseyre AR et al (2016) A comparative study of machine learning techniques for gesture recognition using kinectics. Handbook of research on human-computer interfaces, developments, and applications. IGI Global, Pennsylvania, pp 1–22

    Google Scholar 

  • Langhorne P, Bernhardt J, Kwakkel G (2011) Stroke rehabilitation. Lancet 377(9778):1693–1702

    Article  Google Scholar 

  • Laver K, George S, Thomas S et al (2012) Virtual reality for stroke rehabilitation. Stroke 43(2):e20–e21

    Article  Google Scholar 

  • Li WJ, Hsieh CY, Lin LF, et al (2017) Hand gesture recognition for post-stroke rehabilitation using leap motion. In: Applied system innovation (ICASI), 2017 international conference on; IEEE, pp 386–388

  • McDade E, Kittner S (2009) Ischemic stroke in young adults. Stroke essentials for primary care. Springer, Berlin, pp 123–146

    Chapter  Google Scholar 

  • Okazaki S, Muraoka Y, Suzuki R (2017) Validity and reliability of leap motion controller for assessing grasping and releasing finger movements. J Ergon Technol 17:32–42

    Google Scholar 

  • Placidi G, Cinque L, Polsinelli M et al (2018) Measurements by a leap-based virtual glove for the hand rehabilitation. Sensors 18(3):834

    Article  Google Scholar 

  • Pławiak P, Sośnicki T, Niedźwiecki M et al (2016) Hand body language gesture recognition based on signals from specialized glove and machine learning algorithms. IEEE Trans Ind Inf 12(3):1104–1113

    Article  Google Scholar 

  • Pompeu JE, Alonso TH, Masson IB et al (2014) The effects of virtual reality on stroke rehabilitation: a systematic review. Motricidade 10(4):111–122

    Article  Google Scholar 

  • Regazzoni D, Vitali A, Rizzi C, et al (2018) A method to analyse generic human motion with low-cost mocap technologies. In: ASME 2018 international design engineering technical conferences and computers and information in engineering conference; American Society of Mechanical Engineers Digital Collection

  • Saposnik G, Levin M, Group SORCSW et al (2011) Virtual reality in stroke rehabilitation: a meta-analysis and implications for clinicians. Stroke 42(5):1380–1386

    Article  Google Scholar 

  • Shin JH, Park SB, Jang SH (2015) Effects of game-based virtual reality on health-related quality of life in chronic stroke patients: a randomized, controlled study. Comput Biol Med 63:92–98

    Article  Google Scholar 

  • Teasell RW, Kalra L (2004) What’s new in stroke rehabilitation. Stroke 35(2):383–385

    Article  Google Scholar 

  • Thearling K (2000) Data mining and customer relationships. Building data mining applications for CRM. McGraw Hill, New York, NY

    Google Scholar 

  • Tsoupikova D, Stoykov NS, Corrigan M et al (2015) Virtual immersion for post-stroke hand rehabilitation therapy. Ann Biomed Eng 43(2):467–477

    Article  Google Scholar 

  • Vamsikrishna K, Dogra DP, Desarkar MS (2016) Computer-vision-assisted palm rehabilitation with supervised learning. IEEE Trans Biomed Eng 63(5):991–1001

    Article  Google Scholar 

  • Weichert F, Bachmann D, Rudak B et al (2013) Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5):6380–6393

    Article  Google Scholar 

  • Winstein CJ, Stein J, Arena R et al (2016) Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the american heart association/american stroke association. Stroke 47(6):e98–e169

    Article  Google Scholar 

  • Wu YT, Chen KH, Ban SL et al (2019) Evaluation of leap motion control for hand rehabilitation in burn patients: an experience in the dust explosion disaster in formosa fun coast. Burns 45(1):157–164

    Article  Google Scholar 

  • Yahya M, Shah J, Kadir K, Yusof Z, Khan S, Warsi A (2019) Motion capture sensing techniques used in human upper limb motion: a review. Sensor Rev 39(4):504–511

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the support given by the funding of Erasmus+ K-107 2017–18, and StaRs-Fund 2/2018, University of Bergamo, Italy).

Author information

Authors and Affiliations

  1. Department of Software Engineering, Braude College of Engineering, Karmiel, Israel

    Miri Weiss Cohen

  2. Department of Management, Information and Production Engineering, University of Bergamo, Dalmine, BG, Italy

    Daniele Regazzoni

Authors
  1. Miri Weiss Cohen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniele Regazzoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miri Weiss Cohen .

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Weiss Cohen , M., Regazzoni, D. Hand rehabilitation assessment system using leap motion controller. AI & Soc 35, 581–594 (2020). https://doi.org/10.1007/s00146-019-00925-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-019-00925-8

Keywords

Search

Navigation