Shaun Alexander Macdonald
Frank Pollick
Stephen Anthony Brewster
ABSTRACT
This paper investigates how presenting emotionally resonant vibrotactile stimuli at cool, neutral and warm temperature levels impacts mean ratings for emotional resonance and affective response. Affective vibrotactile stimuli can elicit pleasant or calming responses, making them applicable for emotion regulation. Evoking real-world sensations via emotional resonance can widen their affective range and improve their effectiveness, and allow them to enhance immersive multimodal experiences. Thermotactile cues have been shown to affect emotional responses, but have not been combined with emotionally resonant vibrations to see how they change responses to such cues. This study (n=20) assessed the impact of 3 temperature levels (24℃, 30℃, and 34℃) on 15 emotionally resonant vibrotactile cues and observed if emotionally resonant stimuli exceeded the affective range non-resonant vibrotactile stimuli. The findings suggest that presenting specific resonant vibrations at temperatures that are appropriate for the sensation they evoke can improve emotional resonance and vice versa. In addition, temperature had a positive effect on affective response and emotionally resonant vibrations were found to have a wider affective range than traditional vibrotactile cues. These findings support using emotionally resonant vibrations and thermal cues to elicit desirable emotional responses in emotion regulation and immersive media applications.
Supplemental Material
Available for Download
- Imtiaj Ahmed, Ville Harjunen, Giulio Jacucci, Eve Hoggan, Niklas Ravaja, and Michiel M. Spapé. 2016. Reach out and touch me: Effects of four distinct haptic technologies on affective touch in virtual reality. In ICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction. https://doi.org/10.1145/2993148.2993171Google Scholar
Digital Library
- Moses Akazue, Martin Halvey, Lynne Baillie, and Stephen Brewster. 2016. The effect of thermal stimuli on the emotional perception of images. In Conference on Human Factors in Computing Systems - Proceedings. 4401–4410. https://doi.org/10.1145/2858036.2858307Google ScholarDigital Library
- Akshita, Harini Alagarai Sampath, Bipin Indurkhya, Eunhwa Lee, and Yudong Bae. 2015. Towards Multimodal Affective Feedback. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15 (2015), 2043–2052. https://doi.org/10.1145/2702123.2702288Google ScholarDigital Library
- Y. C.P. Arai, S. Sakakibara, A. Ito, K. Ohshima, T. Sakakibara, T. Nishi, S. Hibino, S. Niwa, and K. Kuniyoshi. 2008. Intra-operative natural sound decreases salivary amylase activity of patients undergoing inguinal hernia repair under epidural anesthesia. Acta Anaesthesiologica Scandinavica 52, 7 (2008), 987–990. https://doi.org/10.1111/j.1399-6576.2008.01649.xGoogle ScholarCross Ref
- Ruben T Azevedo, Nell Bennett, Andreas Bilicki, Jack Hooper, and Fotini Markopoulou. 2017. The calming effect of a new wearable device during the anticipation of public speech. NatureApril(2017), 1–7. https://doi.org/10.1038/s41598-017-02274-2Google Scholar
- Matteo Bianchi, Gaetano Valenza, Antonio Lanata, Alberto Greco, Mimma Nardelli, Antonio Bicchi, and Enzo Pasquale Scilingo. 2017. On the Role of Affective Properties in Hedonic and Discriminant Haptic Systems. International Journal of Social Robotics 9, 1 (2017), 87–95. https://doi.org/10.1007/s12369-016-0371-xGoogle ScholarCross Ref
- Kyung Yun Choi and Hiroshi Ishii. 2020. ambienBeat : Wrist-worn Mobile Tactile Biofeedback for Heart Rate Rhythmic Regulation. TEI 2020 (2020), 17–30. https://doi.org/10.1145/3374920.3374938Google ScholarDigital Library
- Jean Costa, François Guimbretière, Malte Jung, and Tanzeem Choudhury. 2019. BoostMeUp: Improving Cognitive Performance in the Moment by Unobtrusively Regulating Emotions with a Smartwatch. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1–23. https://doi.org/10.1145/3328911Google ScholarDigital Library
- Heather Culbertson, Cara M. Nunez, Ali Israr, Frances Lau, Freddy Abnousi, and Allison M. Okamura. 2018. A social haptic device to create continuous lateral motion using sequential normal indentation. IEEE Haptics Symposium, HAPTICS 2018-March (2018), 32–39. https://doi.org/10.1109/HAPTICS.2018.8357149Google ScholarCross Ref
- Susanne Cutshall, Patricia Anderson, Sharon Prinsen, Laura Wentwoth, Tammy L Olney, Penny K Messner, Karen M Brekke, Thoralf M Sundt Iii, Ryan F Kelly, and Brent A Bauer. 2011. Effect of the Combination of Music and Nature Sounds on Pain and Anxiety in Cardiac Surgical Patients: A Randomized Study. Alternative Therapies in Health and Medicine 17, 4 (2011), 16–24.Google Scholar
- Elaine Czech, Mina Shibasaki, and Keitaro Tsuchiya. 2019. Haptic Remembrance Book Series. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems - CHI ’19. ACM Press, New York, New York, USA, 1–6. https://doi.org/10.1145/3290607.3309685Google ScholarDigital Library
- Dobromir Dobrev & Stephen A. Brewster Graham Wilson. 2016. Hot Under the Collar: Mapping Thermal Feedback to Dimensional. CHI ’16, #chi4good (2016), 4838–4849. https://doi.org/10.1088/0022-3727/46/15/155107Google Scholar
- Martin Halvey, Graham Wilson, Stephen Brewster, and Stephen Hughes. 2012. ” Baby it’s cold outside” the influence of ambient temperature and humidity on thermal feedback. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 715–724.Google ScholarDigital Library
- Lucy Handscomb. 2006. Use of bedside sound generators by patients with tinnitus-related sleeping difficulty: Which sounds are preferred and why?Acta Oto-Laryngologica 126, SUPPL. 556 (2006), 59–63. https://doi.org/10.1080/03655230600895275Google Scholar
- Hikaru Hasegawa, Shogo Okamoto, Ken Ito, and Yoji Yamada. 2019. Affective Vibrotactile Stimuli : Relation between Vibrotactile Parameters and Affective Responses. Transactions of Japan Society of Kansei Engi (2019). https://doi.org/10.5057/ijae.IJAE-D-18-00008Google Scholar
- Alice Haynes, Jonathan Lawry, Christopher Kent, and Jonathan Rossiter. 2021. Feelmusic: Enriching our emotive experience of music through audio-tactile mappings. Multimodal Technologies and Interaction 5, 6 (2021). https://doi.org/10.3390/mti5060029Google Scholar
- Janella Hudson, Rachel Ungar, Laurie Albright, Rifky Tkatch, James Schaeffer, and Ellen R. Wicker. 2020. Robotic Pet Use Among Community-Dwelling Older Adults. The journals of gerontology. Series B, Psychological sciences and social sciences 75, 9 (2020), 2018–2028. https://doi.org/10.1093/geronb/gbaa119Google Scholar
- Gijs Huisman, Aduén Darriba Frederiks, Jan B.F. Van Erp, and Dirk K.J. Heylen. 2016. Simulating affective touch: Using a vibrotactile array to generate pleasant stroking sensations. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 9775. https://doi.org/10.1007/978-3-319-42324-1_24Google Scholar
- Tactile Labs Inc. 2022. Haptuator Markl II Tactile Labs. http://tactilelabs.com/products/haptics/haptuator-mark-ii-v2/.Google Scholar
- Tactile Labs Inc. 2022. Haptuator-Quad-Amplifier. http://tactilelabs.com/wp-content/uploads/2015/02/Haptuator-Quad-Amplifier.pdf.Google Scholar
- Marina Iosifyan and Olga Korolkova. 2019. Emotions associated with different textures during touch. Consciousness and Cognition 71, October 2018 (2019), 79–85. https://doi.org/10.1016/j.concog.2019.03.012Google Scholar
- Ali Israr, Siyan Zhao, Kaitlyn Schwalje, Roberta Klatzky, and Jill Lehman. 2014. Feel Effects. ACM Transactions on Applied Perception 11, 3 (2014), 1–17. https://doi.org/10.1145/2641570 arxiv:1710.03346Google ScholarDigital Library
- James Higgins Jacok Wobbrock, Leah Findlater, Darren Gergle. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures. CHI 2011 (2011), 1–5.Google Scholar
- Lynette A. Jones and Anshul Singhal. 2018. Vibrotactile Pattern Identification in a Multisensory Display. In Haptics: Science, Technology, and Applications, Domenico Prattichizzo, Hiroyuki Shinoda, Hong Z. Tan, Emanuele Ruffaldi, and Antonio Frisoli (Eds.). Springer International Publishing, Cham, 401–412.Google Scholar
- Georgios Karafotias, Akiko Teranishi, Georgios Korres, Friederike Eyssel, Scandar Copti, and Mohamad Eid. 2017. Intensifying emotional reactions via tactile gestures in immersive films. ACM Transactions on Multimedia Computing, Communications and Applications 13, 3(2017). https://doi.org/10.1145/3092840Google ScholarDigital Library
- Chelsea Kelling, Daniella Pitaro, and Jussi Rantala. 2016. Good vibes: The impact of haptic patterns on stress levels. In AcademicMindtrek 2016 - Proceedings of the 20th International Academic Mindtrek Conference. https://doi.org/10.1145/2994310.2994368Google ScholarDigital Library
- Shaun Alexander Macdonald, Stephen Brewster, and Frank Pollick. 2020. Eliciting Emotion with Vibrotactile Stimuli Evocative of Real-World Sensations. In ICMI 2020 - Proceedings ofthe 2020 International Conference on Multimodal Interaction. Utrecht, To Appear. https://doi.org/10.1145/3382507.3418812Google ScholarDigital Library
- Shaun Alexander Macdonald, Euan Freeman, Stephen Brewster, and Frank Pollick. 2021. User Preferences for Calming Affective Haptic Stimuli in Social Settings. ICMI 2021 - Proceedings of the 2021 International Conference on Multimodal Interaction 9781450384, August(2021), 387–396. https://doi.org/10.1145/3462244.3479903Google ScholarDigital Library
- Pardis Miri, Robert Flory, Andero Uusberg, Helen Uusberg, James J. Gross, and Katherine Isbister. 2017. Hapland: A scalable robust Emotion regulation haptic system testbed. In Conference on Human Factors in Computing Systems - Proceedings, Vol. Part F1276. https://doi.org/10.1145/3027063.3053147Google ScholarDigital Library
- Pardis Miri, Emily Jusuf, Andero Uusberg, Horia Margarit, Robert Flory, Katherine Isbister, Keith Marzullo, and James J. Gross. 2020. Evaluating a Personalizable, Inconspicuous Vibrotactile(PIV) Breathing Pacer for In-the-Moment Affect Regulation. In Conference on Human Factors in Computing Systems - Proceedings. Association for Computing Machinery. https://doi.org/10.1145/3313831.3376757Google ScholarDigital Library
- Caitlin Morris, Valdemar Danry, and Pattie Maes. 2022. EmbER: A System for Transfer of Interoceptive Sensations to Improve Social Perception. Association for Computing Machinery (ACM), 277–287. https://doi.org/10.1145/3532106.3533550Google ScholarDigital Library
- mp3gain 2004. Mp3Gain - Home. http://mp3gain.sourceforge.net/.Google Scholar
- Mutsuhiro Nakashige, Hidekazu Tamaki, Minoru Kobayashi, Suguru Higashino, and Yuriko Suzuki. 2009. ”Hiya-Atsu” media: Augmenting digital media with temperature. Conference on Human Factors in Computing Systems - Proceedings (2009), 3181–3186. https://doi.org/10.1145/1520340.1520453Google ScholarDigital Library
- Marianna Obrist, Sriram Subramanian, Elia Gatti, Benjamin Long, and Thomas Carter. 2015. Emotions mediated through mid-air haptics. Conference on Human Factors in Computing Systems - Proceedings 2015-April (2015), 2053–2062. https://doi.org/10.1145/2702123.2702361Google ScholarDigital Library
- JACQUELINE J. OGDEN, DONALD G. LINDBURG, and TERRY L. MAPLE. 2010. The Effects of Ecologically-Relevant Sounds on Zoo Visitors. Curator: The Museum Journal 36, 2 (2010), 147–156. https://doi.org/10.1111/j.2151-6952.1993.tb00787.xGoogle Scholar
- Monica Perusquıa-Hernandez, Marisabel Cuberos Balda, David Antonio Gomez Jauregui, Diego Paez-Granados, Felix Dollack, and Jose Victorio Salazar. 2020. Robot Mirroring: Promoting Empathy with an Artificial Agent by Reflecting the User’s Physiological Affective States. IEEE International Conference on Robot and Human Interactive Communication, 1328–1333.Google Scholar
- James A. Russell. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6(1980), 1161–1178. https://doi.org/10.1037/h0077714Google ScholarCross Ref
- Katri Salminen, Veikko Surakka, Jani Lylykangas, Jukka Raisamo, Rami Saarinen, Roope Raisamo, Jussi Rantala, and Grigori Evreinov. 2008. Emotional and behavioral responses to haptic stimulation. In Conference on Human Factors in Computing Systems - Proceedings. 1555–1562. https://doi.org/10.1145/1357054.1357298Google ScholarDigital Library
- Katri Salminen, Veikko Surakka, Jukka Raisamo, Jani Lylykangas, Johannes Pystynen, Roope Raisamo, Kalle Mäkelä, and Teemu Ahmaniemi. 2011. Emotional responses to thermal stimuli. ICMI’11 - Proceedings of the 2011 ACM International Conference on Multimodal Interaction (2011), 193–196. https://doi.org/10.1145/2070481.2070513Google ScholarDigital Library
- Katri Salminen, Veikko Surakka, Jukka Raisamo, Jani Lylykangas, Roope Raisamo, Kalle Mäkelä, and Teemu Ahmaniemi. 2013. Cold or hot? How thermal stimuli are related to human emotional system?Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7989 LNCS (2013), 20–29. https://doi.org/10.1007/978-3-642-41068-0_3Google ScholarDigital Library
- Hasti Seifi and Karon E. Maclean. 2013. A first look at individuals’ affective ratings of vibrations. 2013 World Haptics Conference, WHC 2013(2013), 605–610. https://doi.org/10.1109/WHC.2013.6548477Google ScholarCross Ref
- Yatiraj Shetty, Shubham Mehta, Diep Tran, Bhavica Soni, and Troy Mcdaniel. 2021. Emotional Response to Vibrothermal Stimuli. Applied Sciences11(2021), 1–16.Google Scholar
- Takanori Shibata, Yukitaka Kawaguchi, and Kazuyoshi Wada. 2009. Investigation on people living with Paro at home effects of sex difference and owners’ animal preference. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication (2009), 1131–1136. https://doi.org/10.1109/ROMAN.2009.5326201Google Scholar
- Sang Won Shim and Hong Z. Tan. 2020. palmscape: Calm and pleasant vibrotactile signals. In International Conference on Human-Computer Interaction. 532–548. https://doi.org/10.1007/978-3-030-49713-2_37Google ScholarDigital Library
- Freesound Team. 2005. About Freesound. https://freesound.org/help/about/.Google Scholar
- Jordan Tewell, Jon Bird, and George Buchana. 2017. The Heat is On: A Temperature Display for Conveying Affective Feedback. In CHI 2017. 1756–1767. https://doi.org/10.1111/j.1744-6171.1994.tb00244.xGoogle ScholarCross Ref
- Masahiko Tsuchiya, A. Asada, K. Ryo, K. Noda, T. Hashino, Y. Sato, E. F. Sato, and Masayasu Inoue. 2003. Relaxing intraoperative natural sound blunts haemodynamic change at the emergence from propofol general anaesthesia and increases the acceptability of anaesthesia to the patient. Acta Anaesthesiologica Scandinavica 47, 8 (2003), 939–943. https://doi.org/10.1034/j.1399-6576.2003.00160.xGoogle ScholarCross Ref
- Muhammad Umair, Corina Sas, Niaz Chalabianloo, and Cem Ersoy. 2021. Exploring Personalized Vibrotactile and Thermal Patterns for Affect Regulation. DIS 2021 - Proceedings of the 2021 ACM Designing Interactive Systems Conference: Nowhere and Everywhere(2021), 891–906. https://doi.org/10.1145/3461778.3462042Google ScholarDigital Library
- Deltcho Valtchanov, Kevin R. Barton, and Colin Ellard. 2010. Restorative Effects of Virtual Nature Settings. Cyberpsychology, Behavior, and Social Networking 13, 5(2010), 503–512. https://doi.org/10.1089/cyber.2009.0308Google ScholarCross Ref
- Patrizia Di Campli San Vito, Stephen Brewster, Frank Pollick, Simon Thompson, Lee Skrypchuk, and Alexandros Mouzakitis. 2020. Purring Wheel: Thermal and Vibrotactile Notifications on the Steering Wheel. ICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction, 461–469. https://doi.org/10.1145/3382507.3418825Google ScholarDigital Library
- Graham Wilson and Stephen A. Brewster. 2017. Multi-Moji: Combining Thermal, Vibrotactile & Visual Stimuli to Expand the Affective Range of Feedback. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17 (2017), 1743–1755. https://doi.org/10.1145/3025453.3025614Google ScholarDigital Library
- Graham Wilson, Gavin Davidson, and Stephen Brewster. 2015. In the heat of the moment: Subjective interpretations of thermal feedback during interaction. Conference on Human Factors in Computing Systems - Proceedings 2015-April, 2063–2072. https://doi.org/10.1145/2702123.2702219Google ScholarDigital Library
- Graham Wilson, Euan Freeman, and Stephen A. Brewster. 2016. Multimodal affective feedback: Combining thermal, vibrotactile, audio and visual signals. ICMI 2016 - Proceedings of the 18th ACM International Conference on Multimodal Interaction (2016), 400–401. https://doi.org/10.1145/2993148.2998522Google ScholarDigital Library
- Mingdi Xu, Takeshi Tachibana, Nana Suzuki, Eiichi Hoshino, Yuri Terasawa, Norihisa Miki, and Yasuyo Minagawa. 2021. The effect of haptic stimulation simulating heartbeats on the regulation of physiological responses and prosocial behavior under stress: The influence of interoceptive accuracy. Biological Psychology 164, August (2021), 108172. https://doi.org/10.1016/j.biopsycho.2021.108172Google ScholarCross Ref
- Steven John Yohanan. 2012. The Haptic Creature Social Human-Robot Interaction through Affective Touch. University of British Columbia Thesis, August (2012), 393.Google Scholar
- Yongjae Yoo, Hojin Lee, Hyejin Choi, and Seungmoon Choi. 2018. Emotional responses of vibrotactile-thermal stimuli: Effects of constant-temperature thermal stimuli. 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017 2018-Janua (2018), 273–278. https://doi.org/10.1109/ACII.2017.8273612Google Scholar
- Yongjae Yoo, Taekbeom Yoo, Jihyun Kong, and Seungmoon Choi. 2015. Emotional responses of tactile icons: Effects of amplitude, frequency, duration, and envelope. IEEE World Haptics Conference, WHC 2015(2015), 235–240. https://doi.org/10.1109/WHC.2015.7177719Google Scholar
- Yizhen Zhou, Aiko Murata, and Junji Watanabe. 2020. The Calming Effect of Heartbeat Vibration. IEEE Haptics Symposium(2020), 677–683.Google Scholar
Index Terms
- The Impact of Thermal Cues on Affective Responses to Emotionally Resonant Vibrations
Recommendations
FeelTheNews: Augmenting Affective Perceptions of News Videos with Thermal and Vibrotactile Stimulation
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing SystemsEmotion plays a key role in the emerging wave of immersive, multi-sensory audience news engagement experiences. Since emotions can be triggered by somatosensory feedback, in this work we explore how augmenting news video watching with haptics can ...
User Preferences for Calming Affective Haptic Stimuli in Social Settings
ICMI '21: Proceedings of the 2021 International Conference on Multimodal InteractionThis paper presents a survey informing a user-first approach to designing calming affective haptic stimuli by eliciting user preferences in different social scenarios. Prior affective haptics research presented users with stimuli and recorded emotional ...
Eliciting Emotion with Vibrotactile Stimuli Evocative of Real-World Sensations
ICMI '20: Proceedings of the 2020 International Conference on Multimodal InteractionThis paper describes a novel category of affective vibrotactile stimuli which evoke real-world sensations and details a study into emotional responses to them. The affective properties of short and abstract vibrotactile waveforms have previously been ...
Comments