Abstract
Autonomous and semi-autonomous robots have been increasingly proposed as social and therapy assistants for neurorehabilitation after traumatic and non-traumatic brain injury. Unfortunately, current robot-patient interactions do not accurately model therapist-patient interactions in task-oriented stroke rehabilitation and therefore may fall short of being clinically effective. In this study, we coded and analyzed 8 videos each showing an occupational therapist interacting with a stroke patient in common activities of daily living settings. We propose that a model of the interaction between a patient and a therapist can be overlaid on a stimulus-response paradigm where the therapist and the patient take on a set of acting states or roles and are motivated to move from one role to another when certain physical or verbal stimuli or cues are sensed and received. We discuss how observed roles and cues can be mapped to current and future examples of robot-patient interactions and implications if such a robot was realized.









Similar content being viewed by others
Data Availibility
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
References
Association American Occupational Therapy (2014) Occupational therapy practice framework: domain and process. Am J Occup Ther 68(1):S1–S48
Arkin RC (1998) Behavior-based robotics. MIT press
Awaad I, Kraetzschmar GK, Hertzberg J (2015) The role of functional affordances in socializing robots. Int J Soc Robot 7(4):421–438
Bayona NA, Bitensky J, Salter K, Teasell R (2005) The role of task-specific training in rehabilitation therapies. Top Stroke Rehabil 12(3):58–65
Beckerle P, Salvietti G, Unal R, Prattichizzo D, Rossi S, Castellini C, Hirche S, Endo S, Amor HB, Ciocarlie M, Mastrogiovanni F, Argall BD, Bianchi M (2017) A human-robot interaction perspective on assistive and rehabilitation robotics. Front Neurorobot 11(MAY):1–6
Bejarano N.C, Maggioni S, De Rijcke L, Cifuentes C.A, Reinkensmeyer D.J (2016) Robot-assisted rehabilitation therapy: recovery mechanisms and their implications for machine design. In: Emerging Therapies in Neurorehabilitation II, pp. 197–223. Springer
Bekey GA (2005) Autonomous robots: from biological inspiration to implementation and control. MIT press
Breazeal C (2004) Social interactions in hri: the robot view. IEEE Trans Syst Man Cybern Part C Appl Rev 34(2):181–186
Brooks DA, Howard AM (2012) Quantifying upper-arm rehabilitation metrics for children through interaction with a humanoid robot. Appl Bionics Biomech 9(2):157–172
Chang WH, Kim YH (2013) Robot-assisted therapy in stroke rehabilitation. J Stroke 15(3):174
Christopher SM, Johnson MJ (2014) Task-oriented robot-assisted stroke therapy of paretic limb improves control in a unilateral and bilateral functional drink task: A case study. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
Chrungoo A, Shirsat P, Johnson M.J (2015) Towards perception driven robot-assisted task-oriented therapy. In: Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR)
Cirstea M, Levin M (2007) Improvement of arm movement patterns and endpoint control depends on type of feedback during practice in stroke survivors. Neurorehabil Neural Repair 21(5):398–411
Costandi M (2014) Rehabilitation: machine recovery. Nature 510(7506):S8–S9
Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16(3):297–334
Devanne M, Nguyen SM (2017) Multi-level motion analysis for physical exercises assessment in kinaesthetic rehabilitation. In: Proceedings of the 17th IEEE-RAS International Conference on Humanoid Robotics (Humanoids), pp 529–534
Duncan PW (1997) Synthesis of intervention trials to improve motor recovery following stroke. Top Stroke Rehabil 3(4):1–20
Eizicovits D, Edan Y, Tabak I, Levy-Tzedek S (2018) Robotic gaming prototype for upper limb exercise: effects of age and embodiment on user preferences and movement. Restor Neurol Neurosci 36(2):261–274
Fasola J, Mataric M (2013) A socially assistive robot exercise coach for the elderly. J Hum Robot Interact 2(2):3–32
Fasola J, Mataric MJ (2012) Using socially assistive human-robot interaction to motivate physical exercise for older adults. Proc IEEE 100(8):2512–2526
Feil-Seifer D, Matarić M.J (2005) Defining socially assistive robotics. In: 9th International Conference on Rehabilitation Robotics (ICORR), pp 465–468. IEEE
Fisher G, Keehn M (2007) Workforce needs and issues in occupational and physical therapy. UIC Midwest Center for Workforce Studies, Chicago, IL
Fitter NT, Hawkes DT, Kuchenbecker KJ (2016) Rhythmic timing in playful human-robot social motor coordination. In: International Conference on Social Robotics, pp 296–305. Springer
Fitter NT, Kuchenbecker KJ (2016) Using imu data to demonstrate hand-clapping games to a robot. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Fitter NT, Kuchenbecker KJ (2017) Hand-clapping games with a baxter robot. In: Proceedings of the Companion of the ACM/IEEE International Conference on Human-Robot Interaction
Fowler M (2004) UML distilled: a brief guide to the standard object modeling language. Addison-Wesley Professional
Radwin G, Yen TY (2005) Multimedia video task analysis (mvta). https://mvta.engr.wisc.edu/
Galvez JA, Budovitch A, Harkema SJ, Reinkensmeyer DJ (2011) Trainer variability during step training after spinal cord injury: implications for robotic gait-training device design. J Rehabil Res Dev 48(2)
Ganesh G, Takagi A, Osu R, Yoshioka T, Kawato M, Burdet E (2014) Two is better than one: physical interactions improve motor performance in humans. Sci Rep 4:3824
Gockley R, Matarić MJ (2006) Encouraging physical therapy compliance with a hands-off mobile robot. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-Robot Interaction
Gomes J, Oliveira SM, Christensen AL (2018) An approach to evolve and exploit repertoires of general robot behaviours. Swarm and Evolutionary Computation
Goodworth A Johnson MJ PM (2019) Chapter 12: physical therapy and rehabilitation
Görer B, Salah AA, Akin HL (2013) A robotic fitness coach for the elderly. In: International Joint Conference on Ambient Intelligence, pp. 124–139. Springer
Guerra J, Uddin J, Nilsen D, Mclnerney J, Fadoo A, Omofuma IB, Hughes S, Agrawal S, Allen P, Schambra HM (2017) Capture, learning, and classification of upper extremity movement primitives in healthy controls and stroke patients. In: Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR)
Gündoğan C (2019) A new physical therapy product design that integrates with water (aquatic therapy) in order to meet needs for patients with multiple sclerosis (ms). Master’s thesis, Izmir Institute of Technology
Guneysu A, Arnrich B (2017) Socially assistive child-robot interaction in physical exercise coaching. In: Proceedings of the 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)
Heinzmann J, Zelinsky A (2003) Quantitative safety guarantees for physical human-robot interaction. Int J Robot Res 22(7–8):479–504
International Clinical Educators(ICE): ICE video library. http://www.icelearningcenter.com/ice-video-library/ (2017)
Itoh K, Miwa H, Nukariya Y, Zecca M, Takanobu H, Roccella S, Carrozza MC, Dario P, Takanishi A (2006) Development of a bioinstrumentation system in the interaction between a human and a robot. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems
Jarrassé N, Charalambous T, Burdet E (2012) A framework to describe, analyze and generate interactive motor behaviors. PLoS ONE 7(11):e49945
Jarrassé N, Sanguineti V, Burdet E (2014) Slaves no longer: review on role assignment for human-robot joint motor action. Adapt Behav 22(1):70–82
Johnson MJ, Christopher SM, Mohan M, Mendonca R (2015) Assessing human-human therapy kinematics for retargeting to human-robot therapy. In: Proceedings of the IEEE International Conference on Rehabilitation Robotics (ICORR)
Johnson MJ, Johnson MA, Sefcik JS, Cacchione PZ, Mucchiani C, Lau T, Yim M (2017) Task and design requirements for an affordable mobile service robot for elder care in an all-inclusive care for elders assisted-living setting. Int J Soc Robot pp 1–20
Johnson MJ, Mendonca R, Venditti N, Jordan A, Barathi S (2016) Do therapists use object affordances to develop functional activities for task-oriented therapy: Implications for robot-assisted task-oriented therapy. Arch Phys Med Rehabil 97(10):e59
Johnson MJ, Mohan M, Mendonca R (2017) A stimulus-response model of therapist-patient interactions in task-oriented stroke therapy can guide robot-patient interactions. In: Proceedings of the RESNA Annual Conference. New Orleans, USA
Johnson MJ, Rai R, Barathi S, Mendonca R, Bustamante-Valles K (2017) Affordable stroke therapy in high-, low-and middle-income countries: from theradrive to rehab cares, a compact robot gym. J Rehabil Assist Technol Eng 4:2055668317708732
Johnson MJ, Sobrepera MJ, Kina E, Mendonca R (2019) Design of an affordable socially assistive robot for remote health and function monitoring and prognostication. Int J Progn Health Manag, 10
Kal E, Prosée R, Winters M, Van Der Kamp J (2018) Does implicit motor learning lead to greater automatization of motor skills compared to explicit motor learning? A systematic review. PLoS ONE 13(9):e0203591
Kearney P, Mcgowan T, Anderson J, Strosahl D (2007) The role of the occupational therapist on the neuro-rehabilitation team
Kleynen M, Braun SM, Rasquin SM, Bleijlevens MH, Lexis MA, Halfens J, Wilson MR, Masters RS, Beurskens AJ (2015) Multidisciplinary views on applying explicit and implicit motor learning in practice: an international survey. PLoS ONE 10(8):e0135522
Lam AWK, Varona-Marin D, Li Y, Fergenbaum M, Kulić D (2016) Automated rehabilitation system: movement measurement and feedback for patients and physiotherapists in the rehabilitation clinic. Hum Comput Interact 31(3–4):294–334
Lang CE, Birkenmeier RL (2014) Upper-extremity task-specific training after stroke or disability: a manual for occupational therapy and physical therapy. AOTA Press
Lang CE, Strube MJ, Bland MD, Waddell KJ, Cherry-Allen KM, Nudo RJ, Dromerick AW, Birkenmeier RL (2016) Dose response of task-specific upper limb training in people at least 6 months poststroke: a phase ii, single-blind, randomized, controlled trial. Ann Neurol 80(3):342–354
Levanon G, Cheng B, Paterra M (2014) The risk of future labor shortages in different occupations and industries in the united states. Bus Econ 49(4):227–243
Liu C, Conn K, Sarkar N, Stone W (2008) Online affect detection and robot behavior adaptation for intervention of children with autism. Trans Robot 24(4):883–896
Lo AC, Guarino PD, Richards LG, Haselkorn JK, Wittenberg GF, Federman DG, Ringer RJ, Wagner TH, Krebs HI, Volpe BT et al (2010) Robot-assisted therapy for long-term upper-limb impairment after stroke. N Engl J Med 362(19):1772–1783
Losey DP, McDonald CG, Battaglia E, O’Malley MK (2018) A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction. Appl Mech Rev, 70(1)
Loureiro RC, Harwin WS, Lamperd R, Collin C (2014) Evaluation of reach and grasp robot-assisted therapy suggests similar functional recovery patterns on proximal and distal arm segments in sub-acute hemiplegia. IEEE Trans Neural Syst Rehabil Eng 22(3):593–602
Loureiro RC, Harwin WS, Nagai K, Johnson M (2011) Advances in upper limb stroke rehabilitation: a technology push. Med Biol Eng Comput 49(10):1103
Luker J, Lynch E, Bernhardsson S, Bennett L, Bernhardt J (2015) Stroke survivors’ experiences of physical rehabilitation: a systematic review of qualitative studies. Arch Phys Med Rehabil 96(9):1698-1708.e10
Luxton DD, Riek LD (2019) Artificial intelligence and robotics in rehabilitation. Handbook of rehabilitation psychology 507–520
Masiero S, Armani M (2011) Upper-limb robot-assisted therapy in rehabilitation of acute stroke patients: focused review and results of new randomized controlled trial. J Rehabil Res Dev 48(4):355
Mastos M, Miller K, Eliasson AC, Imms C (2007) Goal-directed training: linking theories of treatment to clinical practice for improved functional activities in daily life. Clin Rehabil 21(1):47–55
Matarić M, Tapus A, Winstein C, Eriksson J (2009) Socially assistive robotics for stroke and mild tbi rehabilitation. Adv Technol Rehabil 145:249–262
Matarić MJ, Eriksson J, Feil-Seifer DJ, Winstein CJ (2007) Socially assistive robotics for post-stroke rehabilitation. J Neuroeng Rehabil 4(1):5
Matarić MJ (1999) Behavior-based robotics. In: Wilson RA, Keil FC (eds) The MIT Encyclopedia of Cognitive Sciences. MIT Press, Cambridge, pp 74–77
Matsusaka Y, Fujii H, Okano T, Hara I (2009) Health exercise demonstration robot taizo and effects of using voice command in robot-human collaborative demonstration. In: Robot and Human Interactive Communication, 2009. RO-MAN 2009. The 18th IEEE International Symposium on, pp. 472–477. IEEE
Matsusaka Y, Fujii H, Okano T, Hara I (2009) Health exercise demonstration robot taizo and effects of using voice command in robot-human collaborative demonstration. In: Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)
Mavridis N (2015) A review of verbal and non-verbal human-robot interactive communication. Robot Auton Syst 63:22–35
McCarthy C, Butchart J, George M, Kerr D, Kingsley H, Scheinberg A.M, Sterling L (2015) Robots in rehab: Towards socially assistive robots for paediatric rehabilitation. In: Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction, pp. 39–43. ACM
Mehrholz J, Hädrich A, Platz T, Kugler J, Pohl M (2012) Electromechanical and robot-assisted arm training for improving generic activities of daily living, arm function, and arm muscle strength after stroke. Cochrane Syst Rev Interv 6(6)
Min H, Luo R, Zhu J, Bi S et al (2016) Affordance research in developmental robotics: a survey. Trans Cognit Develop Syst 8(4):237–255
Mohan M, Mendonca R, Johnson MJ (2017) Towards quantifying dynamic human-human physical interactions for robot assisted stroke therapy. In: Proceedings of the International Conference on Rehabilitation Robotics (ICORR), pp 913–918. IEEE
Mullick AA, Subramanian SK, Levin MF (2015) Emerging evidence of the association between cognitive deficits and arm motor recovery after stroke: A meta-analysis. Restor Neurol Neurosci 33(3):389–403
Nguyen SM, Tanguy P, Remy-Neris O (2016) Computational architecture of a robot coach for physical exercises in kinaesthetic rehabilitation. In: Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)
Okamura AM, Mataric MJ, Christensen HI (2010) Medical and health-care robotics. IEEE Robot Autom Mag 17(3):26–37
Ovbiagele B, Goldstein LB, Higashida RT, Howard VJ, Johnston SC, Khavjou OA, Lackland DT, Lichtman JH, Mohl S, Sacco RL, Saver JL (2013) Forecasting the future of stroke in the united states: a policy statement from the american heart association and american stroke association. Stroke 44(8):2361–2375
Patton J, Small SL, Zev Rymer W (2008) Functional restoration for the stroke survivor: informing the efforts of engineers. Top Stroke Rehabil 15(6):521–541
Piçarra N, Giger JC (2018) Predicting intention to work with social robots at anticipation stage: Assessing the role of behavioral desire and anticipated emotions. Comput Hum Behav 86:129–146
Reed KB, Peshkin MA (2008) Physical collaboration of human-human and human-robot teams. IEEE Trans Haptics 1(2):108–120
Rensink M, Schuurmans M, Lindeman E, Hafsteinsdottir T (2009) Task-oriented training in rehabilitation after stroke: systematic review. J Adv Nurs 65(4):737–754
Robinson P, Gilabert R (2007) Task complexity, the cognition hypothesis and second language learning and performance. Int Rev Appl Linguist Lang Teach 45(3):161–176
Roetenberg D, Luinge H, Slycke P (2009) Xsens mvn: Full 6dof human motion tracking using miniature inertial sensors. Tech Rep, Xsens Motion Technologies BV
Sawers A, Bhattacharjee T, McKay JL, Hackney ME, Kemp CC, Ting LH (2017) Small forces that differ with prior motor experience can communicate movement goals during human-human physical interaction. J Neuroeng Rehabil 14(1):1–13
Sawers A, Ting LH (2014) Perspectives on human-human sensorimotor interactions for the design of rehabilitation robots. J Neuroeng Rehabil 11(1):142
Schweighofer N, Choi Y, Winstein C, Gordon J (2012) Task-oriented rehabilitation robotics. Am J Phys Med Rehabil 91(11):S270–S279
Seethapathi N, Wang S, Saluja R, Blohm G, Kording KP (2019) Movement science needs different pose tracking algorithms. arXiv preprint arXiv:1907.10226
Shu T, Gao X, Ryoo MS, Zhu SC (2017) Learning social affordance grammar from videos: Transferring human interactions to human-robot interactions. In: 2017 IEEE international conference on robotics and automation (ICRA), pp. 1669–1676. IEEE
Sim DYY, Loo CK (2015) Extensive assessment and evaluation methodologies on assistive social robots for modelling human-robot interaction-a review. Inf Sci 301:305–344
Stanton R, Ada L, Dean CM, Preston E (2015) Feedback Received While Practicing Everyday Activities During Rehabilitation After Stroke: An Observational Study. Physiother Res Int 20(3):166–173
Subramanian SK, Massie CL, Malcolm MP, Levin MF (2010) Does provision of extrinsic feedback result in improved motor learning in the upper limb poststroke? a systematic review of the evidence
Swangnetr M, Kaber DB (2013) Emotional state classification in patient-robot interaction using wavelet analysis and statistics-based feature selection. IEEE Trans Hum Mach Syst 43(1):63–75
Takagi A, Beckers N, Burdet E (2016) Motion plan changes predictably in dyadic reaching. PLoS ONE 11(12):e0167314
Tanguy P, Rémy-Néris O, et al (2016) Computational architecture of a robot coach for physical exercises in kinaesthetic rehabilitation. In: Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp 1138–1143
Tapus A, Mataric MJ (2007) Emulating empathy in socially assistive robotics. In: AAAI Spring Symposium: Multidisciplinary Collaboration for Socially Assistive Robotics, pp 93–96
Tapus A, Tapus C, Mataric M (2009) The role of physical embodiment of a therapist robot for individuals with cognitive impairments. In: Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp 103–107
Tapus A, Ţăpuş C, Matarić MJ (2008) User-robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intel Serv Robot 1(2):169–183
Tiberio L, Cesta A, Olivetti Belardinelli M (2013) Psychophysiological methods to evaluate user’s response in human robot interaction: a review and feasibility study. Robotics 2(2):92–121
Uyanik KF, Calskan Y, Bozcuoglu AK, Yuruten O, Kalkan S, Sahin E(2013) Learning social affordances and using them for planning. In: Proceedings of the Annual Meeting of the Cognitive Science Society, vol. 35
Valles KB, Montes S, de Jesus Madrigal M, Burciaga A, Martínez ME, Johnson MJ (2016) Technology-assisted stroke rehabilitation in mexico: a pilot randomized trial comparing traditional therapy to circuit training in a robot/technology-assisted therapy gym. J Neuroeng Rehabil 13(1):83
Veerbeek JM, Langbroek-Amersfoort AC, Van Wegen EE, Meskers CG, Kwakkel G (2017) Effects of robot-assisted therapy for the upper limb after stroke. Neurorehabil Neural Repair 31(2):107–121
Vegni E, Mauri E, D’Apice M, Moja EA (2010) A quantitative approach to measure occupational therapist-client interactions: A pilot study. Scand J Occup Ther 17(3):217–224
Wallis I, Ingalls T, Rikakis T, Olsen L, Chen Y, Xu W, Sundaram H (2007) Real-time sonification of movement for an immersive stroke rehabilitation environment. In: Proceedings of the 13th International Conference on Auditory Display, pp 26–29
Wang WS, Mendonca R, Kording K, Avery M, Johnson MJ (2019) Towards data-driven autonomous robot-assisted physical rehabilitation therapy. In: Proceedings of the 16th International Conference on Rehabilitation Robotics (ICORR), pp 34–39. IEEE
Weber LM, Stein J (2018) The use of robots in stroke rehabilitation: a narrative review. NeuroRehabilitation 43(1):99–110
World Health Organization and others: Transforming and scaling up health professionals’ education and training: World Health Organization guidelines 2013. World Health Organization (2013)
World Health Organization and others: WHO global disability action plan 2014-2021: Better health for all people with disability. World Health Organization (2015)
Yamanobe N, Wan W, Ramirez-Alpizar IG, Petit D, Tsuji T, Akizuki S, Hashimoto M, Nagata K, Harada K (2017) A brief review of affordance in robotic manipulation research. Adv Robot 31(19–20):1086–1101
Zanfir A, Marinoiu E, Sminchisescu C (2018) Monocular 3d pose and shape estimation of multiple people in natural scenes: The importance of multiple scene constraints. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition pp 2148–2157
Acknowledgements
The Department of Physical Medicine and Rehabilitation at the University of Pennsylvania funded this work. We also want to thank our coders, Lucas Adair and Sarah Laskin. We thank the International Educators Inc. for allowing us to use the videos.
Funding
This study was funded by the University of Pennsylvania, department of physical medicine and rehabilitation (internal grant).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
MJJ has received research grants from the National Institute of Health (NIH), National Science Foundation (NSF), and American Heart Foundation. MJJ and RM are co-founders of a start-up called Recupero Robotics. RM has received funding from Administration for Community Living National Institute on Disability, Independent Living. MM was a graduate student at UPenn at the time this work was done and had no conflict of interest. She is now a PhD student in Germany and still has no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Johnson, M.J., Mohan, M. & Mendonca, R. Therapist-Patient Interactions in Task-Oriented Stroke Therapy can Guide Robot-Patient Interactions. Int J of Soc Robotics 14, 1527–1546 (2022). https://doi.org/10.1007/s12369-022-00881-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12369-022-00881-2