Skip to main content
SpringerLink
Log in

Training of Hand Rehabilitation Using Low Cost Exoskeleton and Vision-Based Game Interface

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Motivating game-based training have the potential to improve therapy for people with neurological impairments. In recent years, the serious games have become extremely useful tools in rehabilitation field. They aim to stimulate the mobility of the body through an immersive experience that puts the user in interactive virtual environment. This paper is concerned about developing a customized augmented reality system for stroke rehabilitation. This will be done through integrating an interactive serious game interface with a hand exoskeleton device. This game-based rehabilitation system allows users to carry out physical rehabilitation therapies using a natural user interface based on Kinect’s skeletal tracking features and the electromyography (EMG) sensor. During game playing, the interactive user interface provides useful real-time feedback information such as the time required to grasp a desired dynamic virtual object, and the assigned score and thus the ability of the proposed system to provide a compensatory action regarding the dynamic behavior of the virtual target. The main goal of the developed virtual environment is to create positive influences on the rehabilitation process. Patient movement information and signals obtained from the developed exoskeleton device are used together to monitor the rehabilitation progress. The developed exoskeleton hand is a 3D printed low cost device suitable for grasping tasks that can be used even for domestic stroke patients. The developed exoskeleton device is not only a mechanical system able to perform the rehabilitation act but also it presents an effective tracking and traceability software solution. The EMG signals measured during hand motion are used to detect the intention of hand opening or closing which in turn will actuate the mechanical structure to accomplish the desired task. Parameters and results of patients’ exercises are stored and analyzed when needed to evaluate patients’ progress. The developed system is tested experimentally and it is able to restore the functions of the upper limb and mainly give patients more motivation to undergo the rehabilitation exercises.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aguilar-Pereyra, J.F.: Design of a reconfigurable robotic system for flexoextension fitted to hand fingers size. In: Applied Bionics and Biomechanics (2016)

  2. Ajoudani, A., Tsagarakis, N., Bicchi, A.: Tele-impedance: teleoperation with impedance regulation using a body–machine interface. Int. J. Robot. Res. 31, 1642–1656 (2012)

    Article  Google Scholar 

  3. Zhang, X., Chen, X., Li, Y., Lantz, V., Wang, K., Yang, J.: A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern. Syst. Hum. 41(6), 1064–1076 (2011)

    Article  Google Scholar 

  4. Baoguo, X., Aiguo, S., Guopu, Z., Jia, L., Guozheng, X., Lizheng, P., Renhuan, Y., Huijun, L., Jianwei, C.: Eeg-modulated robotic rehabilitation system for upper extremity. Biotechnol. Biotechnol. Equip. 32 (3), 795–803 (2018)

    Article  Google Scholar 

  5. Benabdallah, I., Bouteraa, Y., Boucetta, R., Rekik, C.: Kinect-based computed torque control for lynxmotion robotic arm. In: 2015 7th International Conference on Modelling, Identification and Control (ICMIC), pp. 1–6 (2015)

  6. Borboni, A., Mor, M., Faglia, R.: Gloreha-hand robotic rehabilitation: design, mechanical model, and experiments. J. Dyn. Syst. Meas. Control 138(11), 17–28 (2016)

    Article  Google Scholar 

  7. Bouteraa, Y., Abdallah, I.B.: Exoskeleton robots for upper-limb rehabilitation. In: 2016 13th International Multi-conference on Systems, Signals Devices (SSD), pp. 1–6 (2016)

  8. Bouteraa, Y., Ben Abdallah, I.: A gesture-based telemanipulation control for a robotic arm with biofeedback-based grasp. Ind. Robot: Int. J. 44, 575–587 (2017)

    Article  Google Scholar 

  9. Bouteraa, Y., Ghommam, J., Derbel, N., Poisson, G.: Non-linear adaptive synchronisation control of multi-agent robotic systems. Int. J. Syst. Control Commun. 4(1–2), 55–71 (2012)

    Article  Google Scholar 

  10. Bouteraa, Y., Ben Abdallah, I., Ghommam, J.: Task-space region-reaching control for medical robot manipulator. Ind. Robot: Int. J. 67, 629–645 (2018)

    Google Scholar 

  11. Brown, E.V.D., McCoy, S.W., Amber S Fechko, R.P., Gilbertson, T., Moritz, C.T.: Preliminary investigation of an electromyography-controlled video game as a home program for persons in the chronic phase of stroke recovery. Arch. Phys. Med. Rehabil. 95, 1461–1469 (2012)

    Article  Google Scholar 

  12. Cameirao, M., Badia, B., Duarte, E., Frisoli, A., Verschure, P.: The combined impact of virtual reality neurorehabilitation and its interfaces on upper extremity functional recovery in patients with chronic stroke. STROKE 43(10), 2720 (2012)

    Article  Google Scholar 

  13. Cesqui, B., Tropea, P., Micera, S., Krebs, H.I.: Emg-based pattern recognition approach in post stroke robot-aided rehabilitation: a feasibility study. J. Neuroeng. Rehabil. 10(1), 75 (2013)

    Article  Google Scholar 

  14. Chaudhary, A., Raheja, J.L., Das, K., Raheja, S., Intelligent approaches to interact with machines using hand gesture recognition in natural way: a survey. ArXiv e-prints (2013)

  15. Tsai, C.H., Kuo, Y.H., Chu, K.C., Yen, J.C.: Development and evaluation of game-based learning system using the Microsoft Kinect sensor. International Journal of Distributed Sensor Networks 11(7), 498560 (2015)

    Article  Google Scholar 

  16. Cram, J.R., Kasman, G.S., Holtz, J.: Introduction to surface electromyography, 2nd edn. Jones and Bartlett Publishers, Sudbury (2010)

    Google Scholar 

  17. De Luca, C., Donald, L., Mikhail, K., Serge, H.: Filtering the surface emg signal: Movement artifact and baseline noise contamination. J. Biochem. 43(8), 1573–1579 (2010)

    Google Scholar 

  18. Dipietro, L., Sabatini, A., Dario, P.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 38, 461–482 (2008)

    Article  Google Scholar 

  19. Dovat, L., Lambercy, O., Gassert, R., Maeder, T., Milner, T., Leong, T.C., Burdet, E.: HandCARE: a cable-actuated rehabilitation system to train hand function after stroke. IEEE Trans. Neural Syst. Rehabil. Eng. 16(6), 582–591 (2008). https://doi.org/10.1109/TNSRE.2008.2010347

    Article  Google Scholar 

  20. Gabriele, B., Sami, H., Alberto, S.: A critical analysis of a hand orthosis reverse engineering and 3d printing process. In: Applied Bionics and Biomechanics (2016)

  21. Ho, N.S.K., Tong, K.Y., Hu, X.L., Fung, K.L., Wei, X.J., Rong, W., Susanto, E.A.: An emg-driven exoskeleton hand robotic training device on chronic stroke subjects: task training system for stroke rehabilitation. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–5 (2011)

  22. Hyesuk, K., Incheol, K.: Dynamic arm gesture recognition using spherical angle features and hidden markov models. In: Advances in Human-Computer Interaction (2015)

  23. Zannatha, J.M.I., Tamayo, A.J.M., Sánchez, Á.D.G., Delgado, J.E.L., Cheu, L.E.R., Arévalo, W.A.S.: Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation. Comput. Methods Programs Biomed. 112(2), 239–249 (2013)

    Article  Google Scholar 

  24. Díaz, I., Catalan, J.M., Badesa, F.J., Justo, X., Lledo, L.D., Ugartemendia, A., Gil, J.J., Díez, J., García-Aracil, N.: Development of a robotic device for post-stroke home tele-rehabilitation. Advances in Mechanical Engineering 10(1), 1687814017752302 (2018)

    Article  Google Scholar 

  25. Ismail, B.A., Yassine, B., Chokri, R.: Kinect-based sliding mode control for lynxmotion robotic arm. Adv. Human-Comput. Interact. 7921295:1–7921295:10 (2016)

  26. Kiguchi, K.: A study on emg-based human motion prediction for power assist exoskeletons. In: International Symposium on Computational Intelligence in Robotics and Automation, pp. 190–195 (2007)

  27. Kofman, J., Verma, S., Wu, X.: Robot-manipulator teleoperation by markerless vision-based handarm tracking. Int. J. Optomechatronics 1, 331–357 (2007)

    Article  Google Scholar 

  28. Kortier, H.G., Sluiter, V.I., Roetenberg, D., Veltink, P.H.: Assessment of hand kinematics using inertial and magnetic sensors. J. Neuroeng. Rehabil. 11(1), 70 (2014)

    Article  Google Scholar 

  29. Kwon, J.S., Park, M.J., Yoon, I.J., Park, S.H.: Effects of virtual reality on upper extremity function and activities of daily living performance in acute stroke: a double-blind randomized clinical trial. NeuroRehabilitation 31, 379–85 (2012)

    Google Scholar 

  30. Le, C.H., Vander Sloten, J., Hung, L.T., Khanh, L., Soe, S., Zlatov, N., Phuoc, L., Trung, D.P.: Medical reverse engineering applications and methods. In: 2nd International Conference on Innovations, Recent Trends and Challenges in Mechatronics, Mechanical Engineering and New High-Tech Products Development, MECAHITECH, pp. 232–246 (2010)

  31. Lee, G.: Effects of training using video games on the muscle strength, muscle tone, and activities of daily living of chronic stroke patients. J. Phys. Ther. Sci. 25, 595–597 (2013)

    Article  Google Scholar 

  32. Lee, S.W., Wilson, K.M., Lock, B.A., Kamper, D.G.: Subject-specific myoelectric pattern classification of functional hand movements for stroke survivors. IEEE Trans. Neural Syst. Rehabil. Eng. 19(5), 558–566 (2011)

    Article  Google Scholar 

  33. Lee, H.W., Liu, C.H., Chu, K.T., Mai, Y.C., Hsieh, P.C., Hsu, K.C., Tseng, H.C.: Kinect who’s coming—applying kinect to human body height measurement to improve character recognition performance. Smart Science 3, 117–121 (2015)

    Article  Google Scholar 

  34. Mello, R.G.T., Oliveira, L., Nadal, J.: Digital butterworth filter for subtracting noise from low magnitude surface electromyogram. Comput. Methods Prog. Biomed. 87, 28–35 (2007)

    Article  Google Scholar 

  35. Mouawad, M.R., Doust, C.G., Max, M., McNulty, P.: Wii-based movement therapy to promote improved upper extremity function post-stroke: a pilot study. J. Rehabil. Med. 43, 527–533 (2011)

    Article  Google Scholar 

  36. Naik, G.R., Al-Timemy, A.H., Nguyen, H.T.: Transradial amputee gesture classification using an optimal number of semg sensors: an approach using ica clustering. IEEE Trans. Neural Syst. Rehabil. Eng. 24(8), 837–846 (2016)

    Article  Google Scholar 

  37. Naik, G.R., Kumar, D.K.: Twin svm for gesture classification using the surface electromyogram. IEEE Trans. Inf. Technol. Biomed. 14(2), 301–308 (2010)

    Article  Google Scholar 

  38. Phinyomark, A., Phukpattaranont, P., Limsakul, C.: Feature reduction and selection for emg signal classification. Exp. Syst. Appl. 39, 7420–7431 (2012)

    Article  Google Scholar 

  39. Phinyomark, A., Phukpattaranont, P., Limsakul, C.: Fractal analysis features for weak and single-channel upper-limb emg signal. Exp. Syst. Appl. 39, 11156–11163 (2012)

    Article  Google Scholar 

  40. Prochnow, D., Bermúdez i Badia, S., Schmidt, J., Duff, A., Brunheim, S., Kleiser, R., Seitz, R.J., Verschure, P.F.: A functional magnetic resonance imaging study of visuomotor processing in a virtual reality-based paradigm: rehabilitation gaming system. Eur. J. Neurosci. 37, 1441–1447 (2013)

    Article  Google Scholar 

  41. Ferguson, P.W., Dimapasoc, B., Shen, Y., Rosen, J.: Design of a hand exoskeleton for use with upper limb exoskeletons. In: International Symposium on Wearable Robotics, pp. 276–280. Springer, Cham (2018)

    Google Scholar 

  42. Sin, H., Lee, G.: Additional virtual reality training using xbox kinect in stroke survivors with hemiplegia. Am. J. Phys. Med. Rehab./Association of Academic Physiatrists 92, 871–80 (2013)

    Article  Google Scholar 

  43. Sushant, N., Suresh, D., Rajesh, K.S.: Basics and applications of rapid prototyping medical models. Rapid Prototyp. J. 20(3), 256–267 (2014)

    Article  Google Scholar 

  44. Takahashi, C.D., Der-Yeghiaian, L., Le, V., Motiwala, R.R., Cramer, S.C.: Robot-based hand motor therapy after stroke. Brain 131(2), 425–437 (2008)

    Article  Google Scholar 

  45. Turolla, A., Dam, M., Ventura, L., Tonin, P., Agostini, M., Zucconi, C., Kiper, P., Cagnin, A., Piron, L.: Virtual reality for the rehabilitation of the upper limb motor function after stroke: a prospective controlled trial. J. Neuroeng. Rehabil. 10, 85 (2013)

    Article  Google Scholar 

  46. Zhang, X., Yue, Z., Wang, J.: Robotics in lower-limb rehabilitation after stroke. Behav. Neurol. 2017, 13 (2017). Article ID 3731802

    Google Scholar 

  47. Yasemin, C., Abdullah, Y., Mihriban, Y., Ayse Esra, K., Gokhan, K., Alpaslan, G., Gul, G., Huseyin, U., Meliha, K.: Evaluation of invasive and noninvasive methods for the diagnosis of helicobacter pylori infection. Asian Pac. J. Cancer Prev.: APJCP: APJCP 17(12), 5265–5272 (2016)

    Google Scholar 

  48. Yates, M., Kelemen, A., Sik-Lányi, C.: Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Inj. 30, 1–9 (2016)

    Article  Google Scholar 

  49. Yeow, C.H., Baisch, A.T., Talbot, S.G., Walsh, C.J.: Cable-driven finger exercise device with extension return springs for recreating standard therapy exercises. ASME J. Med. Devices 8, 014502 (2014)

    Article  Google Scholar 

  50. Yue, Z., Zhang, X., Wang, J.: Hand rehabilitation robotics on poststroke motor recovery. Behav. Neurol. 2017, 20 (2017). Article ID 3908135

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Prince Sattam Bin Abdulaziz University, Al Kharj, Saudi Arabia

    Yassine Bouteraa & Ahmed M. Elmogy

  2. Digital Research Center of Sfax, CEM Lab-ENIS, University of Sfax, Sfax, Tunisia

    Yassine Bouteraa & Ismail Ben Abdallah

  3. Faculty of Engineering, Tanta University, Tanta, Egypt

    Ahmed M. Elmogy

Authors
  1. Yassine Bouteraa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ismail Ben Abdallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmed M. Elmogy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yassine Bouteraa.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 9.46 MB)

(MP4 411 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bouteraa, Y., Abdallah, I.B. & Elmogy, A.M. Training of Hand Rehabilitation Using Low Cost Exoskeleton and Vision-Based Game Interface. J Intell Robot Syst 96, 31–47 (2019). https://doi.org/10.1007/s10846-018-0966-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-018-0966-6

Keywords

Search

Navigation