Abstract
Robot touch is a vital interaction mode for emotional communication and human mental support in HRI. However, little is known about how robot touch characteristics influence the users’ subjective perception of emotion and physiological reactions. Therefore, a within-subject experiment of robot touches was conducted with the touch type (contact versus grip), length (long versus short), and location (hand versus forearm) as independent variables. The subjective perception was measured with PANAS scales. Electrocardiography (ECG) and functional near-infrared spectroscopy (fNIRS) were utilized to measure the participant’s cardiac autonomic nervous system responses and cerebral central nervous system responses. Results showed that touch type and length jointly affected users’ subjective perception of emotion and cerebral activity, and location affected users’ heart rate variability and cerebral activity. The results suggest that robot short-grip and long-contact behaviors might bring users more positive emotions. Although robot touch on forearms might not induce more positive emotional perception, it might be helpful for emotional regulation and might mobilize more cognitive resources.
Similar content being viewed by others
References
Leo-Liu J (2023) Loving a defiant AI companion? The gender performance and ethics of social exchange robots in simulated intimate interactions. Comput Hum Behav 141:107620. https://doi.org/10.1016/j.chb.2022.107620
Chen TL, King C-HA, Thomaz AL, Kemp CC (2014) An investigation of responses to robot-initiated touch in a nursing context. Int J Social Robot 6(1):141–161. https://doi.org/10.1007/s12369-013-0215-x
Richert A, Schiffmann M, Yuan C (2020) A nursing robot for social interactions and health assessment. Advances in Human Factors in Robots and Unmanned Systems: Proceedings of the AHFE 2019 International Conference on Human Factors in Robots and Unmanned Systems, July 24–28, 2019, Washington DC, USA 10:83–91. https://doi.org/10.1007/978-3-030-20467-9_8
Betlej A (2022) Designing robots for elderly from the perspective of potential end-users: a sociological approach. Int J Environ Res Public Health 19(6):3630. https://doi.org/10.3390/ijerph19063630
Belpaeme T, Kennedy J, Ramachandran A, Scassellati B, Tanaka F (2018) Social robots for education: a review. Sci Rob 3(21):eaat5954. https://doi.org/10.1126/scirobotics.aat5954
Zheng X, Shiomi M, Minato T, Ishiguro H (2021) Modeling the timing and duration of grip behavior to Express emotions for a Social Robot. IEEE Rob Autom Lett 6(1):159–166. https://doi.org/10.1109/lra.2020.3036372
Shiomi M, Zheng X, Minato T, Ishiguro H (2021) Implementation and evaluation of a grip Behavior Model to Express emotions for an Android Robot. Front Robot AI 8:755150. https://doi.org/10.3389/frobt.2021.755150
Zheng X, Shiomi M, Minato T, Ishiguro H (2020) What kinds of Robot’s Touch Will Match expressed emotions? IEEE Rob Autom Lett 5(1):127–134. https://doi.org/10.1109/lra.2019.2947010
Andreasson R, Alenljung B, Billing E, Lowe R (2018) Affective Touch in Human–Robot Interaction: conveying emotion to the Nao Robot. Int J Social Robot 10(4):473–491. https://doi.org/10.1007/s12369-017-0446-3
Willemse CJAM, Toet A, van Erp JBF (2017) Affective and behavioral responses to Robot-initiated Social Touch: toward understanding the opportunities and limitations of Physical Contact in Human–Robot Interaction. Frontiers in ICT 4 https://doi.org/10.3389/fict.2017.00012
Block AE, Kuchenbecker KJ (2018) Emotionally supporting humans through robot hugs. Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction:293-4
Akiyoshi T, Nakanishi J, Ishiguro H, Sumioka H, Shiomi M (2021) A robot that encourages self-disclosure to reduce anger mood. IEEE Rob Autom Lett 6(4):7925–7932. https://doi.org/10.1109/LRA.2021.3102326
Sumioka H, Kumazaki H, Muramatsu T, Yoshikawa Y, Ishiguro H, Higashida H et al (2021) A huggable device can reduce the stress of calling an unfamiliar person on the phone for individuals with ASD. PLoS ONE 16(7):e0254675. https://doi.org/10.1371/journal.pone.0254675
Shiomi M, Hagita N (2021) Audio-Visual Stimuli Change not only Robot’s hug impressions but also its stress-buffering effects. Int J Social Robot 13(3):469–476. https://doi.org/10.1007/s12369-019-00530-1
Zheng X, Shiomi M, Minato T, Ishiguro H (2020) How can Robots make people feel intimacy through Touch? J Robot Mechatron 32(1):51–58. https://doi.org/10.20965/jrm.2020.p0051
DiSalvo C, Gemperle F, Forlizzi J, Montgomery E (2003) The hug: an exploration of robotic form for intimate communication. The 12th IEEE International Workshop on Robot and Human Interactive Communication 2003 Proceedings ROMAN 2003 403–8. https://doi.org/10.1109/ROMAN.2003.1251879
Fitter NT, Kuchenbecker KJ (2020) How does it feel to clap hands with a robot? Int J Social Robot 12(1):113–127. https://doi.org/10.1007/s12369-019-00542-x
Fukuda H, Shiomi M, Nakagawa K, Ueda K (2012) Midas Touch’ in Human-Robot Interaction: evidence from event-related potentials during the Ultimatum game. Int Conf Human-robot Interact. https://doi.org/10.1145/2157689.2157720
Shiomi M, Nakata A, Kanbara M, Hagita N (2021) Robot reciprocation of hugs increases both Interacting Times and Self-disclosures. Int J Social Robot 13(2):353–361. https://doi.org/10.1007/s12369-020-00644-x
Vasara D, Surakka V (2021) Haptic responses to angry and happy faces. Int J Human–Computer Interact 1–11. https://doi.org/10.1080/10447318.2021.1898849
Teyssier M, Bailly G, Pelachaud C, Lecolinet E (2022) Conveying emotions through device-initiated Touch. IEEE Trans Affect Comput 13(3):1477–1488. https://doi.org/10.1109/taffc.2020.3008693
Block AE, Kuchenbecker KJ (2019) Softness, Warmth, and responsiveness improve Robot hugs. Int J Social Robot 11(1):49–64. https://doi.org/10.1007/s12369-018-0495-2
Hertenstein MJ, Holmes R, McCullough M, Keltner D (2009) The communication of emotion via touch. Emotion 9(4):566–573. https://doi.org/10.1037/a0016108
Hertenstein MJ, Keltner D, App B, Bulleit BA, Jaskolka AR (2006) Touch communicates distinct emotions. Emotion 6(3):528. https://doi.org/10.1037/1528-3542.6.3.528
Suvilehtoa JT, Glereana E, Dunbarb RIM, Haria R, Nummenmaaa L (2015) Topography of social touching depends on emotional bonds between humans. Proc Natl Acad Sci U S A 112(48):E6718. https://doi.org/10.1073/pnas.1521810112
Moszkowski RJ, Stack DM, Girouard N, Field TM, Hernandez-Reif M, Diego M (2009) Touching behaviors of infants of depressed mothers during normal and perturbed interactions. Infant Behav Dev 32(2):183–194. https://doi.org/10.1016/j.infbeh.2008.12.009
Ree A, Mayo LM, Leknes S, Sailer U (2019) Touch targeting C-tactile afferent fibers has a unique physiological pattern: a combined electrodermal and facial electromyography study. Biol Psychol 140:55–63. https://doi.org/10.1016/j.biopsycho.2018.11.006
Salminen K, Surakka V, Raisamo J, Lylykangas J, Raisamo R, Mäkelä K et al (2013) Cold or hot? how thermal stimuli are related to human emotional system? Haptic and Audio Interaction Design: 8th International Workshop, HAID 2013, Daejeon, Korea, April 18–19, 2013, Revised Selected Papers 8:20–9
Ogawa K, Nishio S, Koda K, Balistreri G, Watanabe T, Ishiguro H (2011) Exploring the natural reaction of Young and aged person with Telenoid in a Real World. J Adv Comput Intell Intell Inf 15(5):592–597
Turkle S, Breazeal C, Dasté O (2006) B S Encounters with Kismet and Cog: Children Respond to Relational Artifacts. Digital media: Transformations in human communication 120
Shiomi M, Nakata A, Kanbara M, Hagita N (2017) A hug from a robot encourages prosocial behavior. 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) 418–423. https://doi.org/10.1109/ROMAN.2017.8172336
Cooney MD, Nishio S, Ishiguro H (2012) Recognizing affection for a touch-based interaction with a humanoid robot. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems 1420–1427. https://doi.org/10.1109/iros.2012.6385956
Blythe E, Garrido L, Longo MR (2023) Emotion is perceived accurately from isolated body parts, especially hands. Cognition 230:105260. https://doi.org/10.1016/j.cognition.2022.105260
Law T, Malle BF, Scheutz M (2021) A touching connection: how observing robotic Touch can affect Human Trust in a Robot. Int J Social Robot. https://doi.org/10.1007/s12369-020-00729-7
Willis FN, Briggs LF (1992) Relationship and touch in public settings. J Nonverbal Behav 16(1):55–63
McGlone F, Wessberg J, Olausson H (2014) Discriminative and affective touch: sensing and feeling. Neuron 82(4):737–755. https://doi.org/10.1016/j.neuron.2014.05.001
Morrison I, Löken LS, Minde J, Wessberg J, Perini I, Nennesmo I et al (2011) Reduced C-afferent fibre density affects perceived pleasantness and empathy for touch. Brain 134(4):1116–1126. https://doi.org/10.1093/brain/awr011
Rolls ET (2010) The affective and cognitive processing of touch, oral texture, and temperature in the brain. Neurosci Biobehavioral Reviews 34(2):237–245. https://doi.org/10.1016/j.neubiorev.2008.03.010
Bickmore TW, Fernando R, Ring L, Schulman D (2010) Empathic touch by Relational agents. IEEE Trans Affect Comput 1(1):60–71. https://doi.org/10.1109/t-affc.2010.4
Lange CG (1885) The mechanism of the emotions. The classical psychologists: 672–84
Izard CE (1991) The psychology of emotions. Springer Science & Business Media
Lazarus RS, Folkman S (1984) Stress, appraisal, and coping. Springer publishing company
Ekman P (1992) An argument for basic emotions. Cognition & Emotion 6(3–4):169–200. https://doi.org/10.1080/02699939208411068
Ekman P, Friesen WV, O’sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K et al (1987) Universals and cultural differences in the judgments of facial expressions of emotion. J Personal Soc Psychol 53(4):712
Jamil MH, Park W, Eid M (2020) Emotional responses to watching and touching 3d emotional face in a virtual environment. Virtual Reality 25(2):553–564. https://doi.org/10.1007/s10055-020-00473-3
Watson D, Tellegen A (1985) Toward a consensual structure of mood. Psychol Bull 98(2):219. https://doi.org/10.1037/0033-2909.98.2.219
Block AE (2021) HuggieBot: an interactive hugging Robot with Visual and Haptic Perception. ETH Zurich, pp 125–229. https://doi.org/10.3929/ethz-b-000508212
Kreibig SD (2010) Autonomic nervous system activity in emotion: a review. Biol Psychol 84(3):394–421. https://doi.org/10.1016/j.biopsycho.2010.03.010
Shiota MN, Neufeld SL, Yeung WH, Moser SE, Perea EF (2011) Feeling good: autonomic nervous system responding in five positive emotions. Emotion 11(6):1368–1378. https://doi.org/10.1037/a0024278
Golland Y, Keissar K, Levit-Binnun N (2014) Studying the dynamics of autonomic activity during emotional experience. Psychophysiology 51(11):1101–1111. https://doi.org/10.1111/psyp.12261
Christian C, Evelyne V-M, Georges D, Andre D (1997) Autonomic nervous system response patterns specificity to basic emotions. J Auton Nerv Syst 62(1–2):45–57. https://doi.org/10.1016/S0165-1838(96)00108-7
Hidehiro N, Shinichi F, Satoshi O, Tsutomu M, Hiroshi K (2009) Emotion-related changes in heart rate and its variability during performance and perception of music. Ann N Y Acad Sci 1169(1):359–362. https://doi.org/10.1111/j.1749-6632.2009.04788.x
Wascher CA (2021) Heart rate as a measure of emotional arousal in evolutionary biology. Philosophical Trans Royal Soc B 376(1831):20200479
Brosschot JF, Thayer JF (2003) Heart rate response is longer after negative emotions than after positive emotions. Int J Psychophysiol 50(3):181–187. https://doi.org/10.1016/S0167-8760(03)00146-6
Spodick DH, Raju P, Bishop RL, Rifkin RD (1992) Operational definition of normal sinus heart rate. Am J Cardiol 69(14):1245–1246. https://doi.org/10.1016/0002-9149(92)90947-W
Levenson RW, Ekman P, Friesen WV (1990) Voluntary facial action generates emotion-specific autonomic nervous system activity. Psychophysiology 27(4):363–384
Hubert W, de Jong-Meyer R (1990) Psychophysiological response patterns to positive and negative film stimuli. Biol Psychol 31(1):73–93. https://doi.org/10.1016/0301-0511(90)90079-C
Kleiger RE, Stein PK, Bigger JT Jr (2005) Heart rate variability: measurement and clinical utility. Ann Noninvasive Electrocardiol 10(1):88–101. https://doi.org/10.1111/j.1542-474X.2005.10101.x
Montano N, Porta A, Cogliati C, Costantino G, Tobaldini E, Casali KR et al (2009) Heart rate variability explored in the frequency domain: a tool to investigate the link between heart and behavior. Neurosci Biobehavioral Reviews 33(2):71–80. https://doi.org/10.1016/j.neubiorev.2008.07.006
Electrophysiology TFESCNASP (1996) Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation 93(5):1043–1065. https://doi.org/10.1161/01.CIR.93.5.1043
Zupan M, Buskas J, Altimiras J, Keeling LJJP (2016) behavior Assessing positive emotional states in dogs using heart rate and heart rate variability. 155:102–11. https://doi.org/10.1016/j.physbeh.2015.11.027
Triscoli C, Croy I, Steudte-Schmiedgen S, Olausson H, Sailer U (2017) Heart rate variability is enhanced by long-lasting pleasant touch at CT-optimized velocity. Biol Psychol 128:71–81. https://doi.org/10.1016/j.biopsycho.2017.07.007
Christou-Champi S, Farrow TF, Webb TL (2015) Automatic control of negative emotions: evidence that structured practice increases the efficiency of emotion regulation. Cogn Emot 29(2):319–331. https://doi.org/10.1080/02699931.2014.901213
Iadecola C (2017) The neurovascular unit coming of age: a journey through neurovascular coupling in health and disease. Neuron 96(1):17–42. https://doi.org/10.1016/j.neuron.2017.07.030
Friston KJ, Fletcher P, Josephs O, Holmes A, Rugg M, Turner R (1998) Event-related fMRI: characterizing differential responses. NeuroImage 7(1):30–40
Damasio AR, Grabowski TJ, Bechara A, Damasio H, Ponto LLB, Parvizi J et al (2000) Subcortical and cortical brain activity during the feeling of self-generated emotions. Nat Neurosci 3(10):1049–1056
Hu X, Zhuang C, Wang F, Liu YJ, Im CH, Zhang D (2019) fNIRS evidence for recognizably different positive emotions. Front Hum Neurosci 13:120. https://doi.org/10.3389/fnhum.2019.00120
Himichi T, Fujita H, Nomura M (2015) Negative emotions impact lateral prefrontal cortex activation during theory of mind: an fNIRS study. Soc Neurosci 10(6):605–615. https://doi.org/10.1080/17470919.2015.1017112
Chick CF, Rolle C, Trivedi HM, Monuszko K, Etkin A (2020) Transcranial magnetic stimulation demonstrates a role for the ventrolateral prefrontal cortex in emotion perception. Psychiatry Res 284:112515. https://doi.org/10.1016/j.psychres.2019.112515
Ahern GL, Schwartz GE (1985) Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis. Neuropsychologia 23(6):745–755. https://doi.org/10.1016/0028-3932(85)90081-8
Wheeler RE, Davidson RJ, Tomarken AJ (1993) Frontal brain asymmetry and emotional reactivity: a biological substrate of affective style. Psychophysiology 30(1):82–89. https://doi.org/10.1111/j.1469-8986.1993.tb03207.x
Herrington JD, Mohanty A, Koven NS, Fisher JE, Stewart JL, Banich MT et al (2005) Emotion-modulated performance and activity in left dorsolateral prefrontal cortex. Emotion 5(2):200. https://doi.org/10.1037/1528-3542.5.2.200
Etkin A, Egner T, Kalisch R (2011) Emotional processing in anterior cingulate and medial prefrontal cortex. Trends Cogn Sci 15(2):85–93. https://doi.org/10.1016/j.tics.2010.11.004
Grafton ST, Arbib MA, Fadiga L, Rizzolatti G (1996) Localization of grasp representations in humans by positron emission tomography: 2. Observation compared with imagination. Exp Brain Res 112:103–111
Binkofski F, Amunts K, Stephan KM, Posse S, Schormann T, Freund HJ et al (2000) Broca’s region subserves imagery of motion: a combined cytoarchitectonic and fMRI study. Hum Brain Mapp 11(4):273–285. https://doi.org/10.1002/1097-0193(200012)11:4>273::AID-HBM40<3.0.CO;2-0
Binkofski F, Buccino G (2004) Motor functions of the Broca’s region. Brain Lang 89(2):362–369. https://doi.org/10.1016/S0093-934X(03)00358-4
Gardecki A, Podpora M (2017) Experience from the operation of the Pepper humanoid robots. 2017 Progress in Applied Electrical Engineering (PAEE):1–6. https://doi.org/10.1109/PAEE.2017.8008994
Faul F, Erdfelder E, Lang A-G, Buchner A (2007) G* power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 39(2):175–191
Dahlbäck N, Jönsson A, Ahrenberg L (1993) Wizard of Oz studies—why and how. Knowl Based Syst 6(4):258–266
Hoffmann L, Kramer NC (2021) The persuasive power of robot touch. Behavioral and evaluative consequences of non-functional touch from a robot. PLoS ONE 16(5):e0249554. https://doi.org/10.1371/journal.pone.0249554
Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: the PANAS scales. J Personal Soc Psychol 54(6):1063–1070. https://doi.org/10.1037/0022-3514.54.6.1063
Makowski D, Pham T, Lau ZJ, Brammer JC, Lespinasse F, Pham H et al (2021) NeuroKit2: a Python toolbox for neurophysiological signal processing. Behav Res Methods 53(4):1689–1696. https://doi.org/10.3758/s13428-020-01516-y
Singh AK, Okamoto M, Dan H, Jurcak V, Dan I (2005) Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI. NeuroImage 27(4):842–851. https://doi.org/10.1016/j.neuroimage.2005.05.019
Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119(2):593–609. https://doi.org/10.1093/brain/119.2.593
Hou X, Zhang Z, Zhao C, Duan L, Gong Y, Li Z et al (2021) NIRS-KIT: a MATLAB toolbox for both resting-state and task fNIRS data analysis. Neurophotonics 8(1):010802. https://doi.org/10.1117/1.NPh.8.1.010802
Strangman G, Culver JP, Thompson JH, Boas DA (2002) A quantitative comparison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation. NeuroImage 17(2):719–731. https://doi.org/10.1006/nimg.2002.1227
Cope M, Delpy DT (1988) System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Med Biol Eng Comput 26(3):289–294
Fishburn FA, Ludlum RS, Vaidya CJ, Medvedev AV (2019) Temporal derivative distribution repair (TDDR): a motion correction method for fNIRS. NeuroImage 184:171–179. https://doi.org/10.1016/j.neuroimage.2018.09.025
Björnsdotter M, Olausson H (2011) Vicarious responses to social touch in posterior insular cortex are tuned to pleasant caressing speeds. J Neurosci 31(26):9554–9562. https://doi.org/10.1523/JNEUROSCI.0397-11.2011
Huisman G (2017) Social touch technology: a survey of haptic technology for social touch. IEEE Trans Haptics 10(3):391–408. https://doi.org/10.1109/TOH.2017.2650221
Balconi M, Grippa E, Vanutelli ME (2015) What hemodynamic (fNIRS), electrophysiological (EEG) and autonomic integrated measures can tell us about emotional processing. Brain Cogn 95:67–76. https://doi.org/10.1016/j.bandc.2015.02.001
Turnbull A, Wang H, Murphy C, Ho N, Wang X, Sormaz M et al (2019) Left dorsolateral prefrontal cortex supports context-dependent prioritisation of off-task thought. Nat Commun 10(1):3816
Murray LJ, Ranganath C (2007) The dorsolateral prefrontal cortex contributes to successful relational memory encoding. J Neurosci 27(20):5515–5522. https://doi.org/10.1523/JNEUROSCI.0406-07.2007
Lu X, Li T, Xia Z, Zhu R, Wang L, Luo YJ et al (2019) Connectome-based model predicts individual differences in propensity to trust. Hum Brain Mapp 40(6):1942–1954. https://doi.org/10.1002/hbm.24503
Duncan J, Owen AM (2000) Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends Neurosci 23(10):475–483. https://doi.org/10.1016/S0166-2236(00)01633-7
Saygin ZM, Osher DE, Koldewyn K, Reynolds G, Gabrieli JD, Saxe RR (2012) Anatomical connectivity patterns predict face selectivity in the fusiform gyrus. Nat Neurosci 15(2):321–327. https://doi.org/10.1038/nn.3001
Acknowledgements
This work was supported by the National Natural Science Foundation of China (Grant No. 72071035). No conflict of interest exists in submitting this paper, and all authors approve it for publication. We are grateful to all the experimental participants for this study. Furthermore, we are genuinely pleased to extend our gratitude to editors and reviewers for their valuable work.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Guo, F., Fang, C., Li, M. et al. How do Robot Touch Characteristics Impact Users’ Emotional Responses: Evidence from ECG and fNIRS. Int J of Soc Robotics 16, 619–634 (2024). https://doi.org/10.1007/s12369-024-01110-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12369-024-01110-8