Skip to main content

Advertisement

SpringerLink
Log in

Robot sensor system for supervised rehabilitation with real-time feedback

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Numerous assistive robot approaches have been proposed to rehabilitate individuals with impaired upper-limb function. However, to the best of our knowledge, none of these are fully supervised by the robotic system. In this research, we intend to use a robot as a tool to provide robot-guided supervised rehabilitation. A humanoid robot, NAO, was used for this purpose. To demonstrate rehabilitation exercises with NAO, a library of recommended rehabilitation exercises involving the shoulder (abduction/adduction, vertical flexion/extension, and internal/external rotation), and elbow (flexion/extension) joint movements were created. An Xbox Kinect sensor was used to analyze the subject upper arm movement during rehabilitation. For this purpose, a complete geometric solution was developed to find a unique inverse kinematic solution of human upper-arm from the Kinect data. A control algorithm was developed in MATLAB for the proposed robot guided supervised rehabilitation protocol. Experimental results show that the NAO and Kinect sensor can effectively be used to supervise and guide the subjects in performing active rehabilitation exercises for the shoulder and elbow joint movements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Ali A, Sundaraj K, Ahmad B, Ahamed N, Islam A (2012) Gait disorder rehabilitation using vision and non-vision based sensors: a systematic review. Bosnian Journal of Basic Medical Sciences 12(3):193–202

    Article  Google Scholar 

  2. Bäck I, Makela K, Kallio J (2013) Robot-Guided Exercise Program for the Rehabilitation of Older Nursing Home Residents. Annals of Long-Term Care: Clinical Care and Aging 21:38–41

    Google Scholar 

  3. Bai J, Song A, Li H (2019) Design and analysis of cloud upper limb rehabilitation system based on motion tracking for post-stroke patients. Appl Sci 9(8):1620

    Article  Google Scholar 

  4. Bakhti KKA, Laffont I, Muthalib M, Froger J, Mottet D (2018) Kinect-based assessment of proximal arm non-use after a stroke. Journal of neuroengineering and rehabilitation 15(1):104

    Article  Google Scholar 

  5. Brainin M, Teuschl Y, Kalra L (2007) Acute treatment and long-term management of stroke in developing countries. The Lancet Neurology 6(6):553–561

    Article  Google Scholar 

  6. Laisi Cai, Ye Ma, Shuping Xiong, and Yanxin Zhang (2019). “Validity and reliability of upper limb functional assessment using the microsoft kinect V2 Sensor,” Applied Bionics and Biomechanics, vol. 2019, Article ID 7175240, 14 pages

  7. M Capecci, MG Ceravolo, F D’Orazio, F Ferracuti, S Iarlori, G Lazzaro, S Longhi, L Romeo, F Verdini (2015). A tool for home-based rehabilitation allowing for clinical evaluation in a visual markerless scenario, in: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 8034–8037.

  8. M Capecci, MG Ceravolo, F D’Orazio, F Ferracuti, S Iarlori, G Lazzaro, S Longhi, L Romeo, F Verdini (2015), A tool for home-based rehabilitation allowing for clinical evaluation in a visual markerless scenario, in: 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 8034–8037.

  9. Carrillo M, Felip JB, Knight S, Scheinberg A, Wise L, Sterling L, McCarthy C (2018) Adapting a general-purpose social robot for paediatric rehabilitation through in situ design. ACM Transactions on Human-Robot Interaction (THRI) 7(1):1–30

    Article  Google Scholar 

  10. L Ciabattoni, F Ferracuti, S Iarlori, S Longhi, L Romeo (2016). A novel computer vision based e-rehabilitation system: From gaming to therapy support, in: IEEE International Conference on Consumer Electronics (ICCE), pp. 43–44.

  11. Craig, JJ (2005). Introduction to robotics: mechanics and control. Upper Saddle River, N.J., Pearson/Prentice Hall.

  12. Devanne M, Rémy-Néris O, Le Gals-Garnett B, Kermarrec G, Thepaut A (2018) A co-design approach for a rehabilitation robot coach for physical rehabilitation based on the error classification of motion errors. In: 2018 Second IEEE International Conference on Robotic Computing (IRC), pp. 352–357. IEEE

  13. Dutta T (2012) Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace. Appl Ergon 43(4):645–649

    Article  Google Scholar 

  14. Fan J, Beuscher L, Newhouse PA, Mion LC, Sarkar N (2016) “A robotic coach architecture for multi-user human-robot interaction (RAMU) with the elderly and cognitively impaired,” 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN), New York, pp 445–450. https://doi.org/10.1109/ROMAN.2016.7745157

  15. Görer B, Salah AA, Akın HL (2013) A robotic fitness coach for the elderly. In: In International Joint Conference on Ambient Intelligence, pp. 124–139. Springer, Cham

    Google Scholar 

  16. Görer B, Salah AA, Akın HL (2017) An autonomous robotic exercise tutor for elderly people. Auton Robot 41(3):657–678

    Article  Google Scholar 

  17. D Gouaillier et al. (2009). “Mechatronic design of NAO humanoid,” 2009 IEEE International Conference on Robotics and Automation, Kobe, pp. 769–774.doi: https://doi.org/10.1109/ROBOT.2009.5152516

  18. Guidali M, Duschau-Wicke A, Broggi S, Klamroth-Marganska V, Nef T, Reiner RA (2011) A robotic system to train activities of daily living in a virtual environment. Med Biol Eng Comput 49(10):1213–1223

    Article  Google Scholar 

  19. Hossein Mousavi Hondori and Maryam Khademi (2014), “A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation,” Journal of Medical Engineering, vol. 2014, Article ID 846514, 16 pages

  20. Hu XL, Tong KY, Song R, Zheng XJ, Lui KH, Leung WWF, Ng S, Au-Yeung SSY (2009) Quantitative evaluation of motor functional recovery process in chronic stroke patients during robot-assisted wrist training. J Electromyogr Kinesiol 19(4):639–650

    Article  Google Scholar 

  21. Islam MR, Spiewak C, Rahman MH, Fareh R (2017) A brief review on robotic exoskeletons for upper extremity rehabilitation to find the gap between research Porotype and commercial type. Adv Robot Autom 6:177. https://doi.org/10.4172/2168-9695.1000177

    Article  Google Scholar 

  22. Kiguchi K, Rahman MH, Sasaki M, Teramoto K (2008) Development of a 3DOF mobile exoskeleton robot for human upper-limb motion assists. Robot Auton Syst 56(8):678–691

    Article  Google Scholar 

  23. Krebs HI, Hogan N, Aisen ML, Volpe BT (1998) Robot-aided Neurorehabilitation. IEEE transactions on rehabilitation engineering: a publication of the IEEE Engineering in Medicine and Biology Society 6(1):75–87

    Article  Google Scholar 

  24. Lei Q, Ji-Xiang D, Zhang H-B, Ye S, Chen D-S (2019) A survey of vision-based human action evaluation methods. Sensors 19(19):4129

    Article  Google Scholar 

  25. Lotfi A, Langensiepen C, Yahaya SW (2018) Socially assistive robotics: Robot exercise trainer for older adults. Technologies 6(1):32

    Article  Google Scholar 

  26. Lum PS, Burgar CG, Shor PC (2004) Evidence for improved muscle activation patterns after retraining of reaching movements with the MIME robotic system in subjects with post-stroke hemiparesis. IEEE Transactions on Neural Systems and Rehabilitation Engineering 12(2):186–194

    Article  Google Scholar 

  27. Mobini A, Behzadipour S, Saadat Foumani M (2014) Accuracy of Kinect's skeleton tracking for upper body rehabilitation applications. Disability and Rehabilitation: Assistive Technology 9(4):344–352

    Google Scholar 

  28. Ningjia, Y, Feng, D, Yudi, W, Chuang, L, Tan, JTC, Binbin, X, Jin, Z (2013). A study of the human-robot synchronous control system based on skeletal tracking technology. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2191–2196

  29. Otten A, Voort C, Stienen A, Aarts R, Asseldonk Ev, Kooij HVD (2015) LIMPACT:a hydraulically powered self-aligning upper limb exoskeleton. IEEE/ASME Transactions on Mechatronics 20(5):2285–2298. https://doi.org/10.1109/TMECH.2014.2375272

    Article  Google Scholar 

  30. Perry JC, Rosen J, Burns S (2007) Upper-limb powered exoskeleton design. IEEE/ASME Transactions on Mechatronics 12(408–417):408–417

    Article  Google Scholar 

  31. Piron L, Turolla A, Tonin P, Piccione F, Lain L, Dam M (2008) Satisfaction with care in post-stroke patients undergoing a telerehabilitation programme at home. J Telemed Telecare 14(5):257–260

    Article  Google Scholar 

  32. Rahman MH, Rahman MJ, Cristobal OL, Saad M, Kenné JP, Archambault PS (2015) Development of a whole arm wearable robotic exoskeleton for rehabilitation and to assist upper limb movements. Robotica CJO 33(1):19–39. https://doi.org/10.1017/S0263574714000034

    Article  Google Scholar 

  33. Scherer M, Unterbrunner A, Riess B, Kafka P (2016) Development of a system for supervised training at home with Kinect V2. Procedia Engineering 147:466–471 ISSN 1877-7058

    Article  Google Scholar 

  34. Semblantes PA, Andaluz VH, Lagla J, Chicaiza FA, Acurio A (2018) Visual feedback framework for rehabilitation of stroke patients. Informatics in Medicine Unlocked 13:41–50

    Article  Google Scholar 

  35. Sucar LE, Azcarate G, Leder RS et al (2009) Gesture therapy: a vision-based system for arm rehabilitation afer stroke. In: Fred A, Filipe J, Gamboa H (eds) Biomedical Engineering Systems and Technologies. Springer, Berlin, pp 531–540

    Google Scholar 

  36. Tao Y, Hu H, Zhou H (2007) Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation. Int J Robot Res 26(6):607–624

    Article  Google Scholar 

  37. Terven JR, Córdova-Esparza DM (2016) Kin2. A Kinect 2 toolbox for MATLAB. Sci Comput Program 130:97–106

    Article  Google Scholar 

  38. A Thobbi and W Sheng (2010). “Imitation learning of arm gestures in presence of missing data for humanoid robots”, IEEE-RAS Int. Conf. On Humanoid Robots, pp. 92–97, Nashville, TN, USA

  39. J Venugopalan, C Cheng, TH Stokes, and MD Wang (2013). “Kinect-based rehabilitation system for patients with traumatic brain injury,” in Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4625–4628

  40. Wang F, Tang C, Ou Y, Xu Y (2012) “A Real-Time Human Imitation System”, World Congress on Intelligent Control and Automation, pp. 3692–3697, Beijing

  41. Winter, DA (1990). Biomechanics and motor control of human movement, 2nd ed. New York: J. Wiley, xvi, 277 p.

  42. Xu J, De’Aira GB, Chen Y-P, Howard A (2018) Robot therapist versus human therapist: Evaluating the effect of corrective feedback on human motor performance. In: 2018 International Symposium on Medical Robotics (ISMR), pp. 1–6. IEEE

  43. Xu Q, Huang G, Yu M, Guo Y (2020) Fall prediction based on key points of human bones. Physica A: Statistical Mechanics and its Applications 540:123205

    Article  MathSciNet  Google Scholar 

  44. Yang C, Kerr A, Stankovic V, Stankovic L, Rowe P, Cheng S (2016) Human upper limb motion analysis for post-stroke impairment assessment using video analytics. IEEE Access 4:650–659. https://doi.org/10.1109/ACCESS.2016.2523803

    Article  Google Scholar 

  45. Zannatha JMI, Tamayo AJM, Sánchez ÁDG, Delgado JEL, Cheu LER, Arévalo WAS (2013) Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation. Comput Methods Prog Biomed 112(2):239–249 ISSN 0169-2607

    Article  Google Scholar 

  46. Zannatha JMI, Tamayo AJM, Sanchez ADG, Delgado JEL, Cheu LER, Arevalo WAS (2013) Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation. Comput Methods Prog Biomed 112(2):239–249

    Article  Google Scholar 

  47. W Zhao, R Lun, DD Espy, MA Reinthal (2014). Rule based real-time motion assessment for rehabilitation exercises, in: IEEE Symposium on Computational Intelligence in Healthcare and e-health (CICARE), pp. 133–140.

Download references

Author information

Authors and Affiliations

  1. BioRobotics Lab, Mechanical Engineering Department, University of Wisconsin-Milwaukee, Room # USR-281, 115 E Reindl Way, Milwaukee, WI, 53212, USA

    Md Assad Uz Zaman, Md Rasedul Islam & Mohammad Habib Rahman

  2. Clement J. Zablocki VA Medical Center, Milwaukee, WI, USA

    Katie Schultz

  3. Department of Physical Medicine & Rehabilitation, Medical College of Wisconsin (MCW), Milwaukee, WI, USA

    Erin McGonigle

  4. Occupational Science & Technology Department, University of Wisconsin-Milwaukee, Milwaukee, WI, USA

    Inga Wang

Authors
  1. Md Assad Uz Zaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Md Rasedul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Habib Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Katie Schultz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Erin McGonigle
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Inga Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Md Assad Uz Zaman.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(WMV 10910 kb)

ESM 2

(WMV 10872 kb)

ESM 3

(WMV 38404 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Assad Uz Zaman, M., Islam, M.R., Rahman, M.H. et al. Robot sensor system for supervised rehabilitation with real-time feedback. Multimed Tools Appl 79, 26643–26660 (2020). https://doi.org/10.1007/s11042-020-09266-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09266-x

Keywords

Search

Navigation