Abstract
This paper presents a system called REAPP, a virtual environment aiming at supporting upper-limb robotic neurorehabilitation for post-stroke patients. In the last few years different systems using either or both virtual environments and robots have been developed. However, such advanced systems often lack attention on the physical and psychological conditions perceived by the patient, who should be the key focus in therapies. In this context, REAPP aims not to be just an entertaining interface but also a means to investigate patients’ feelings toward the therapy and to enhance their self-perception while performing the rehabilitation exercises. Before and after the rehabilitation session the patients are inquired about their emotional state, their physical condition and a self-evaluation of the performance is done. While performing the exercises patients look at a 2D/3D interface which has the aim of enhancing their self-awareness by showing them, through an avatar, the movements they are doing. Widgets and a virtual assistant provide additional feedback about the quality of the movements which is computed through a specifically developed arm model that can be customized on the residual capabilities of each patient.







Similar content being viewed by others
References
Mendis S (2013) Stroke disability and rehabilitation of stroke: World Health Organization perspective. Int J Stroke 8(1):3–4
Janssen H, Ada L, Bernhardt J, MvElduff P, Pollack M, Nilsson M, Spratt N (2014) Physical, cognitive and social activity levels of stroke patients undergoing rehabilitation within a mixed rehabilitation unit. Clin Rehabil 28(1):91–101
Pekna M, Pekny M, Nilsson M (2012) Modulation of Neural Plasticity as a Basis for Stroke Rehabilitation. Stroke 43:2819–2828
Chang WH, Kim Y-H (2013) Robot-assisted therapy in stroke rehabilitation. J Stroke 3(15):174–181
Lohse KR, Hilderman CGE, Cheung KL, Tatla S, Van der Loos HFM (2014) Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. Plos One 9(3):e93318
Fluet GG, Deutsch JE (2013) Virtual reality for sensorimotor rehabilitation post-stroke: the promise and current state of the field. Curr Phys Med Rehabil Rep 1(1):9–20
Novak D, Nagke A, Keller U, Riener R (2014) Increasing motivation in robot-aided arm rehabilitation with competitive and cooperative gameplay. J Neuroeng Rehabil 11:64
Riener R, Nef T, Colombo G (2005) Robot-aided neurorehabilitation of the upper extremities. Med Biol Eng Comput 43:2–10
Coote S, Stokes E (2003) Robot mediated therapy: attitudes of patients and therapists toward the first prototype of the GENTLE/s system. Technol Disabil 15:27–34
Lum PS, Burgar CG, Shor SC, Majmundar M, Van der Loos M (2002) Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper limb motor function after stroke. Arch Phys Med Rehabil 83(7):952–959
Maclean N, Piund P (2000) A critical review of the concept of patient motivation in the literature on physical reahabilitation. Soc Sci Med 50:495–506
Borghese NA, Pirovano M, Lanzi PL, Wüest S, de Bruin ED (2013) Computational intelligence and game design for effective at home rehabilitation. Games Health J 2(2):81–88
Bertilsson AS, Ranner M, Von Kich L, Eriksson G, Johansson U, Ytterberg C, Guidetti S, Tham K (2014) A client-centered ADL intervention: three-month follow-up of a randomized controlled trial. Scand J Occup Ther 1–15
Pastor I, Hayes HA, Bamberg SJM (2012) A feasibility study of an upper limb rehabilitation system using Kinect and computer games. In: 34th annual international conference of the IEEE EMBS, San Diego, California USA
Mirelman A, Bonato P, Deutsch DE (2009) Effects of training with a robot-virtual reality system compared with a robot alone on the gait of individuals after stroke. Stroke 40(1):169–174
Brütsch K, Koenig A, Zimmerli L, Mérillat-Koeneke S, Riener R, Jäncke L, van Hedel HJ, Meyer-Heim A (2011) Virtual reality for enhancement of robot-assisted gait training in children with central gait disorders. J Rehabil Med 43(6):493–499
Shah N, Basteris A, Amirabdollahian F (2014) Design parameters in multimodal games for rehabilitation. Games Health J 3(1):13–20
Malosio M, Pedrocchi N, Molinari Tosatti L (2010) Robot-assisted upper-limb rehabilitation platform. In: Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, Nara, Japan
Scano A, Caimmi M, Malosio M, Pedrocchi N, Molinari Tosatti L, Molteni F (2013) Upper limb robotic rehabilitation: Treatment customization. Gait Posture 37(1):S13–S14
Caimmi M, Carda S, Giovanzana C, Maini E, Sabatini AM, Smania N, Molteni F (2008) Using kinematic analysis to evaluate constraint-induced movement therapy in chronic stroke patients. Neurorehabil Neural Repair 22:31–39
Aprile I, Rabuffetti M, Padua L, Di Sipio E, Simbolotti C, Ferrarin M (2014) Using kinematic analysis to evaluate constraint-induced movement therapy in chronic stroke patients. BioMed Res Int
Redaelli C, Caimmi M, Molteni F (2014) Percezione dell’interazione uomo/robot nella riabilitazione assistita: il punto di vista del paziente. In: \(42^{\circ }\) congresso nazionale SIMFER, Torino
Lin KC, Chen YT, Huang PC (2014) Effect of mirror therapy combined with somatosensory stimulation on motor recovery and daily function in stroke patients: A pilot study. J Formos Med Assoc 113(7):422–428
Zucconi C, Valt V, Agostini M, Turolla A, Tonin P, Piron L (2011) Assessment of a virtual teacher feedback for the recovery of the upper limb after stroke. Ital J Physiother
Albert SJ, Kesselring J (2011) Neurorehabilitation of stroke. J Neurol 259(5):817–832
Cherney LR, van Vuuren S (2012) Telerehabilitation, virtual therapists, and acquired neurologic speech and language disorders. In: seminars in speech and language
Parker J, Mawson S, Mountain G, Nasr N, Zheng H (2014) Stroke patients’ utilisation of extrinsic feedback from computer-based technology in the home: a multiple case study realistic evaluation. BMC Med Inform Decis Mak 14(1):14–46
Ekman P, Friesen W, Hager J (2002) Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Mountain View
Smith VA, DeVellis BM, Kalet A, Roberts JC, DeVellis RF (2005) Encouraging patient adherence: primary care physicians’ use of verbal compliance-gaining strategies in medical interviews. Patient Educ Couns 57:62–76
Koenig A, Omlin X, Zimmerli L, Sapa M, Krewer C, Bollinger M, Müller F, Riener R (2011) Psychological state estimation from physiological recordings during robot-assisted gait rehabilitation. J Rehabil Res Dev 48(4):367–385
Zangiacomi A, Redaelli C, Valentini F, Bernardelli G (2014) Design of interaction in a Virtual Environment for post-stroke rehabilitation: a cognitive perspective. In: 5th IEEE cogInfoCom conference, Vietri sul Mare, Italy
Cherkassky T, Oksenberg A, Froom P, Ring H (2003) Sleep-related breathing disorders and rehabilitation outcome of stroke patients: a prospective study. Am J Phys Med Rehabil 82(6):452–455
Dib S, Ramos AR, Douglas WM, Rundek T (2005) Sleep and stroke. Period Biol 114(3):369–375
Spalletta G, Pasini A, Costa A, De Angelis D, Ramundo N, Paolucci S, Caltagirone C (2001) Alexithymic features in stroke: effects of laterality and gender. Psychosom Med 63(6):944–950
Lane RD, Sechrest L, Reidel R, Weldon V, Kaszniak A (1996) Impaired verbal and nonverbal emotion recognition in alexithymia. Psychosom Med 58(3):203–210
Yuvaraj RA, Murugappan MA, Norlinah MI, Sundaraj KA, Khairiyah MB (2013) Review of emotion recognition in stroke patients. Dement Geriatr Cogn Disord 36:179–196
Wiles R, Ashburn A, Payne S, Murphy C (2002) Patients’ expectations of recovery following stroke: a qualitative study. Disabil Rehabil 24(16):841–850
Lange B, Change CY, Suma E, Newman B, Rizzo AS, Bolas M (2011) Development and evaluation of low cost game- based balance rehabilitation tool using the Microsoft Kinect sensor. In: 33rd annual international conference of the IEEE EMBS, Massachusetts USA, Boston
Kitzmüller G, Häggström T, Asplund K (2013) Living an unfamiliar body: the significance of the long-term influence of bodily change on the perception of self after stroke. Med Health Care Philos 29:16–19
Prochnow D, Bermùz i Badia S, Schmidt J, Duff A, Brunheim S, Kleiser R, Seitz RJ, Verschure PFMJ (2013) A functional magnetic resonance imaging study of visuomotor processing in a virtual reality - based paradigm: rehabilitation gaming system. Eur J Neurosci 37:1441–1447
Rizzolatti G, Fabbri-Destro M, Cattaneo L (2009) Mirror neuron and their clinical relevance. Nat Clin Pract Neurol 5:24–34
Stefan K, Classen J, Celnik P, Cohen LG (2008) Cuncurrent action observation modulates practise-induced motor memory formation. Eur J Neurosci 27(3):730–738
Ertelt D, Small S, Solodkin A, Dettmers C, McNamara A, Binkofski F, Buccino G (2007) Action observation has a positive impact on rehabilitation of motor deficits after stroke. Neuroimage 36(2):164–173
Kil-Soo S, Hongki K, Eung KS (2011) What if your avatar looks like you? Dualcongruity perspectives for avatar use. MIS Q 35(3):711–729
Changsoo K, Sang-Gun L, Minchoel K (2012) I became an attractive person in the virtual world: users’ identification with virtual communities and avatars. Comput Hum Behav 28(5):1663– 1669
Vasalou A, Joinson AN (2009) Me, myself and I: The role of interactional context on self-presentation through avatars. Comput Hum Behav 25(2):510–520
Teng C-I (2010) Customization, immersion satisfaction, and online gamer loyalty. Comput Hum Behav 26(6):1547–1554
Li DD, Kien Liau A, Khoo A (2013) layer-Avatar Identification in video gaming: concept and measurement. Comput Hum Behav 29(1):257–263
Nowak KL (2013) Choosing Buddy Icons that look like me or represent my personality: using Buddy Icons for social presence. Comput Hum Behav 29(4):1456–1464
Bailey R, Wise K, Bolls P (2009) How avatar customizability affects children’s arousal and subjective presence during junk food-sponsored online video games. Cyber psychol Behav 12(3):277–283
Biocca F (1997) The Cyborg’s dilemma: progressive embodiment in Virtual Environments. J Comput Med Commun 3:(2)
Youjeong K, Shyam Sundar S (2012) Visualizing ideal self vs. actual self through avatars: impact on preventive health outcomes. Comput Hum Behav 28(4):1356–1364
Jin SAA (2010) I feel more connected to the physically deal Mini Me than the mirror-image Mini Me: theoretical implications of the “Malleable Self” for speculations on the effects of avatar creation on Avatar-Self Connection in Wii. Cyberpsychol Behav Soc Netw 13(5):567–570
Keppel CC, Crowe SF (2000) Changes to body image and self-esteem following stroke in young adults. Neuropsychol Rehabil 10(1):15–31
Hole E, Stubbs B, Roskell C, Soundy A (2014) The Patient’s Experience of the Psychosocial Process That Influences Identity following Stroke Rehabilitation: A Metaethnography. Sci World J 2014:349151
Krakauer JW (2006) Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr Opin Neurol 19(1):84–90
Liao WW, Wu CY, Hsieh YW, Lin KC, Chang WY (2011) Effects of robot-assisted upper limb rehabilitation on daily function and real-world arm activity in patients with chronic stroke: a randomized controlled trial. Clin Rehabil 26(2):111–120
Thielman G (2010) Rehabilitation of reaching poststroke: a randomized pilot investigation of tactile versus aditory feedback for trunk control. J Neurol Phys Ther 34(3):138–144
Arya KN, Pandian S, Verma R, Garg KR (2011) Movement therapy induced neural reorganization and motor recovery in stroke: a review. J Bodyw Mov Ther 15(4):528–537
McPhail SM, Nalder E, Hill AM, Haines TP (2013) Physiotherapists have accurate expectations of their patients’ future health-related quality of life after first assessment in a subacute rehabilitation setting. Biomed Res Int 2013:340371
Baranyi P, Csapo A (2010) Cognitive Infocommunications: CogInfoCom. In: Proceedings of 11th IEEE international symposium on computational intelligence and informatics, Budapest
Baranyi P, Csapo A (2012) Definition and synergies of Cognitive Infocommunications. Acta Polytechnica Hungarica 9(1):67–83
Wright RW (1997) Affective computing. MIT Press, Cambridge
Csapo A, Wersenyi G (2013) Overview of auditory representations in human-machine interfaces. ACM Comput Surv 46(2):1–23
Blattner MM, Sumikava DA, Greenberg RM (1989) Earcons and icons: their structure and common principles design. Hum Comput Interact 4:11–44
Rosati G, Rodà A, Avanzini F, Masier S (2013) On the role of the auditory feedback in robot-assisted movement training after stroke: review of the literature. Comput Intell Neurosci 2013:586138
Levin MF, Kleim JA, Wolf SL (2008) What do motor “recovery” and “compensation” mean in patients following stroke? Neurorehabil Neural Repair 23(4):313–319
Taati B, Wang R, Huq R, Snoek J, Mihailidis A (2012) Vision-based posture assessment to detect and categorize compensation during robotic rehabilitation therapy. In: the fourth IEEE RAS/EMBS internationalconference on biomedical robotics and biomechatronics, Roma, Italy
Colombo R, Sterpi I, Mazzone A, Delconte C, Pisano F (2013) Robot-aided neurorehabilitation in sub-acute and chronic stroke: does spontaneous recovery have a limited impact on outcome? NeuroRehabilitation 33(4):621–629
Cirstea MC, Levin MF (2000) Compensatory strategies for reaching in stroke. Brain 123:940–953
Yeunga L, Chengb KC, Fonga C, Leea WC, Tonga K-Y (2013) Evaluation of the Microsoft Kinect as a clinical assessment tool of body sway. Gait Posture 40(4):532–538
Hu MC, Chen CW, Cheng WH, Chang CH, Lai JH, Wu JL (2014) Real-time human movement retrieval and assessment with kinect sensor. In: Proceedings of IEEE transactions cybernetics
Rikakis T (2011) Utilizing media arts principles for developing effective interactive neurorehabilitation systems. In: 33rd annual international conference of the IEEE EMBS, Boston, Massachusetts, USA
Malosio M, Caimmi M, Legnani G, Molinari Tosatti L (2014) LINarm: a low-cost variable stiffness device for upper-limb rehabilitation. In: 2014 IEEE/RSJ international conference on intelligent robots and systems, Chicago
Acknowledgments
The presented work is carried out within RIPRENDO@Home Project, regional research project funded inside the Framework Agreement between Regione Lombardia and National Research Council, D.G.R. n. 3728 - 11/07/2012.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mottura, S., Fontana, L., Arlati, S. et al. A virtual reality system for strengthening awareness and participation in rehabilitation for post-stroke patients. J Multimodal User Interfaces 9, 341–351 (2015). https://doi.org/10.1007/s12193-015-0184-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12193-015-0184-5