research-article

Reach out and touch me: effects of four distinct haptic technologies on affective touch in virtual reality

Authors:

Ville Harjunen,

Giulio Jacucci,

Michiel M. SpapéAuthors Info & Claims

ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Pages 341 - 348

https://doi.org/10.1145/2993148.2993171

Published: 31 October 2016 Publication History

Abstract

Virtual reality presents an extraordinary platform for multimodal communication. Haptic technologies have been shown to provide an important contribution to this by facilitating co-presence and allowing affective communication. However, the findings of the affective influences rely on studies that have used myriad different types of haptic technology, making it likely that some forms of tactile feedback are more efficient in communicating emotions than others. To find out whether this is true and which haptic technologies are most effective, we measured user experience during a communication scenario featuring an affective agent and interpersonal touch in virtual reality. Interpersonal touch was simulated using two types of vibrotactile actuators and two types of force feedback mechanisms. Self-reports of subjective experience of the agent’s touch and emotions were obtained. The results revealed that, regardless of the agent’s expression, force feedback actuators were rated as more natural and resulted in greater emotional interdependence and a stronger sense of co-presence than vibrotactile touch.

References

[1]

Ahmed, I., Harjunen, V., Jacucci, G., Ravaja, N., and Spapé, M.M. Total Immersion: Designing for Affective Symbiosis in a Virtual Reality Game with Haptics, Biosensors and Emotive Agents. In Symbiotic Interaction. Springer, 2016.

[2]

App, B., McIntosh, D.N., Reed, C.L., and Hertenstein, M.J. Nonverbal channel use in communication of emotion: How may depend on why. Emotion 11, 3 (2011), 603.

[3]

Bailenson, J.N., Yee, N., Brave, S., Merget, D., and Koslow, D. Virtual interpersonal touch: Expressing and recognizing emotions through haptic devices. Human– Computer Interaction 22, 3 (2007), 325–353.

Digital Library

[4]

Cabibihan, J.-J., Zheng, L., and Cher, C.K.T. Affective teletouch. In Social robotics. Springer, 2012, 348–356.

Digital Library

[5]

Courgeon, M., Buisine, S., and Martin, J.-C. Impact of expressive wrinkles on perception of a virtual character’s facial expressions of emotions. Intelligent Virtual Agents, Springer (2009), 201–214.

Digital Library

[6]

Courgeon, M., Martin, J.-C., and Jacquemin, C. Marc: a multimodal affective and reactive character. Proceedings of the 1st Workshop on AFFective Interaction in Natural Environments, (2008).

[7]

Ekman, P. Facial expression and emotion. American psychologist 48, 4 (1993), 384.

[8]

Ekman, P. and Friesen, W.V. Constants across cultures in the face and emotion. Journal of personality and social psychology 17, 2 (1971), 124.

[9]

Ekman, P. and Friesen, W.V. Measuring facial movement. Environmental psychology and nonverbal communication. Human Sciences Press, San Francisco, 1976.

[10]

van Erp, J.B. and Spapé, M.M. Distilling the underlying dimensions of tactile melodies. Proceedings of Eurohaptics, Citeseer (2003), 111–120.

[11]

Gallace, A. and Spence, C. The science of interpersonal touch: an overview. Neuroscience & Biobehavioral Reviews 34, 2 (2010), 246–259.

[12]

Guenter, B., Grimm, C.M., and Malvar, H.S. Method and system for capturing and representing 3D geometry, color and shading of facial expressions and other animated objects. Google Patents, 2000.

[13]

Haans, A. and IJsselsteijn, W. Mediated social touch: a review of current research and future directions. Virtual Reality 9, 2–3 (2006), 149–159.

Digital Library

[14]

Haans, A. and Usselsteijn, W.A. The virtual Midas touch: Helping behavior after a mediated social touch. Haptics, IEEE Transactions on 2, 3 (2009), 136–140.

Digital Library

[15]

Hannig, G. and Deml, B. Efficient bimodal haptic weight actuation. In Haptics: Generating and Perceiving Tangible Sensations. Springer, 2010, 3–10.

Digital Library

[16]

Hertenstein, M.J., Holmes, R., McCullough, M., and Keltner, D. The communication of emotion via touch. Emotion 9, 4 (2009), 566.

[17]

Hertenstein, M.J., Keltner, D., App, B., Bulleit, B.A., and Jaskolka, A.R. Touch communicates distinct emotions. Emotion 6, 3 (2006), 528.

[18]

Hoggan, E., Brewster, S.A., and Johnston, J. Investigating the effectiveness of tactile feedback for mobile touchscreens. Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (2008), 1573– 1582.

Digital Library

[19]

Hoggan, E., Stewart, C., Haverinen, L., Jacucci, G., and Lantz, V. Pressages: augmenting phone calls with nonverbal messages. Proceedings of the 25th annual ACM symposium on User interface software and technology, ACM (2012), 555–562.

Digital Library

[20]

Izard, C.E. The face of emotion. (1971).

[21]

Jacucci, G., Spagnolli, A., Freeman, J., and Gamberini, L. Symbiotic interaction: a critical definition and comparison to other human-computer paradigms. In Symbiotic Interaction. Springer, 2014, 3–20.

[22]

Jylhä, A., Hsieh, Y.-T., Orso, V., Andolina, S., Gamberini, L., and Jacucci, G. A Wearable Multimodal Interface for Exploring Urban Points of Interest. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, ACM (2015), 175–182.

Digital Library

[23]

Lisetti, C., Nasoz, F., LeRouge, C., Ozyer, O., and Alvarez, K. Developing multimodal intelligent affective interfaces for tele-home health care. International Journal of Human-Computer Studies 59, 1 (2003), 245–255.

Digital Library

[24]

Mathiak, K. and Weber, R. The Fun of Gaming: Measuring the Human Experience of Media Enjoyment—Subproject/partner no. 5 “The fMRI-based approach”. Research proposal and study guide for the European Commission’s NEST program. Coordination Dr. Niklas Ravaja, Helsinki School of Economics. Finland). Unpublished manuscript, RWTH Aachen University, Germany & Michigan State University, East Lansing, 2006.

[25]

Mueller, F., Vetere, F., Gibbs, M.R., Kjeldskov, J., Pedell, S., and Howard, S. Hug over a distance. CHI’05 extended abstracts on Human factors in computing systems, ACM (2005), 1673–1676.

Digital Library

[26]

Nakanishi, H., Tanaka, K., and Wada, Y. Remote handshaking: touch enhances video-mediated social telepresence. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2014), 2143– 2152.

Digital Library

[27]

Nam, C.S., Richard, P., Yamaguchi, T., and Bahn, S. Does Touch Matter?: The Effects of Haptic Visualization on Human Performance, Behavior and Perception. International Journal of Human-Computer Interaction 30, 11 (2014), 839–841.

[28]

Robles-De-La-Torre, G. The importance of the sense of touch in virtual and real environments. Ieee Multimedia 13, 3 (2006), 24–30.

Digital Library

[29]

Sherrick, C.E. A scale for rate of tactual vibration. The Journal of the Acoustical Society of America 78, 1 (1985), 78–83.

[30]

Shimoga, K.B. A survey of perceptual feedback issues in dexterous telemanipulation. II. Finger touch feedback. Virtual Reality Annual International Symposium, 1993., 1993 IEEE, IEEE (1993), 271–279.

Digital Library

[31]

Spapé, M.M., Hoggan, E.E., Jacucci, G., and Ravaja, N. The meaning of the virtual Midas touch: An ERP study in economic decision making. Psychophysiology 52, 3 (2015), 378–387.

[32]

Takahashi, K., Mitsuhashi, H., Murata, K., Norieda, S., and Watanabe, K. Improving shared experiences by haptic telecommunication. Biometrics and Kansei Engineering (ICBAKE), 2011 International Conference on, IEEE (2011), 210–215.

Digital Library

[33]

Van Erp, J.B. Guidelines for the use of vibro-tactile displays in human computer interaction. Proceedings of Eurohaptics, (2002), 18–22.

[34]

Wang, R., Quek, F., Teh, J.K., Cheok, A.D., and Lai, S.R. Design and evaluation of a wearable remote social touch device. International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ACM (2010), 45.

Digital Library

[35]

Zmigrod, S., Spapé, M., and Hommel, B. Intermodal event files: integrating features across vision, audition, taction, and action. Psychological Research PRPF 73, 5 (2008), 674–684.

Cited By

Zhang ZLi JYan ZNishida JPeng H(2024)JetUnit: Rendering Diverse Force Feedback in Virtual Reality Using Water JetsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676440(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676440
Chauvette LLeprohon ÉPerron-Houle LPintat VDelnavaz AVoix JSharp A(2024)Multichannel Vibrotactile Glove: Validation of a New Device Designed to Sense VibrationsIEEE Transactions on Haptics10.1109/TOH.2024.347574017:4(913-923)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TOH.2024.3475740
Iriarte NEzcurdia IElizondo SIrisarri JHemmerling DOrtiz AMarzo A(2024)Contactless Electrostatic Piloerection for Haptic SensationsIEEE Transactions on Haptics10.1109/TOH.2023.326988517:2(140-151)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TOH.2023.3269885
Show More Cited By

Index Terms

Reach out and touch me: effects of four distinct haptic technologies on affective touch in virtual reality
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

The Experience of Social Touch in Multi-User Virtual Reality
VRST '20: Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology

We present user study results on virtual body contact experience in a two-user VR scenario, in which participants performed different touches with a research assistant. The interaction evoked different emotional reactions in perceived relaxation, ...
Haptic technologies for direct touch in virtual reality
SIGGRAPH '16: ACM SIGGRAPH 2016 Courses

Virtual reality (VR) is experiencing a renaissance thanks to technological progress in computer graphics and the commercial breakthroughs in head-mounted display and tracking technologies. Fully immersive VR requires virtual touch of comparably high ...
HexTouch: Affective Robot Touch for Complementary Interactions to Companion Agents in Virtual Reality
VRST '20: Proceedings of the 26th ACM Symposium on Virtual Reality Software and Technology

There is a growing need for social interaction in Virtual Reality (VR). Current social VR applications enable human-agent or interpersonal communication, usually by means of visual and audio cues. Touch, which is also an essential method for affective ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

October 2016

605 pages

ISBN:9781450345569

DOI:10.1145/2993148

General Chairs:
Yukiko I. Nakano
Seikei University, Japan
,
Elisabeth André
Augsburg University, Germany
,
Toyoaki Nishida
Kyoto University, Japan
,
Program Chairs:
Louis-Philippe Morency
Carnegie Mellon University, USA
,
Carlos Busso
University of Texas at Dallas, USA
,
Catherine Pelachaud
ISIR, France / University of Paris6, France

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMI '16

Sponsor:

SIGCHI

ICMI '16: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 12 - 16, 2016

Tokyo, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,197
Total Downloads

Downloads (Last 12 months)180
Downloads (Last 6 weeks)25

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang ZLi JYan ZNishida JPeng H(2024)JetUnit: Rendering Diverse Force Feedback in Virtual Reality Using Water JetsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676440(1-15)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676440
Chauvette LLeprohon ÉPerron-Houle LPintat VDelnavaz AVoix JSharp A(2024)Multichannel Vibrotactile Glove: Validation of a New Device Designed to Sense VibrationsIEEE Transactions on Haptics10.1109/TOH.2024.347574017:4(913-923)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TOH.2024.3475740
Iriarte NEzcurdia IElizondo SIrisarri JHemmerling DOrtiz AMarzo A(2024)Contactless Electrostatic Piloerection for Haptic SensationsIEEE Transactions on Haptics10.1109/TOH.2023.326988517:2(140-151)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TOH.2023.3269885
Valori IJung MFairhurst M(2024)Social touch to build trustComputers in Human Behavior10.1016/j.chb.2023.108121153:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.chb.2023.108121
Jacucci GBellucci AAhmed IHarjunen VSpape MRavaja N(2024)Haptics in social interaction with agents and avatars in virtual reality: a systematic reviewVirtual Reality10.1007/s10055-024-01060-628:4Online publication date: 14-Nov-2024
https://dl.acm.org/doi/10.1007/s10055-024-01060-6
Kounalaki MSimou IKomninos A(2023)Pseudo-haptic and Self-haptic Feedback During VR Text EntryProceedings of the 2nd International Conference of the ACM Greek SIGCHI Chapter10.1145/3609987.3610002(1-8)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3609987.3610002
Askari SHarjunen VSpapé MHaans ARavaja NIJsselsteijn W(2023)Receiving a Mediated Touch From Your Partner vs. a Male Stranger: How Visual Feedback of Touch and Its Sender Influence Touch ExperienceIEEE Transactions on Affective Computing10.1109/TAFFC.2021.308518514:2(1044-1055)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TAFFC.2021.3085185
Ahmed IHarjunen VJacucci GRavaja NRuotsalo TSpape M(2023)Touching Virtual Humans: Haptic Responses Reveal the Emotional Impact of Affective AgentsIEEE Transactions on Affective Computing10.1109/TAFFC.2020.303813714:1(331-342)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TAFFC.2020.3038137
Olugbade THe LMaiolino PHeylen DBianchi-Berthouze N(2023)Touch Technology in Affective Human–, Robot–, and Virtual–Human Interactions: A SurveyProceedings of the IEEE10.1109/JPROC.2023.3272780111:10(1333-1354)Online publication date: Oct-2023
https://doi.org/10.1109/JPROC.2023.3272780
Liu QGhodrat SHuisman GJansen K(2023)Shape memory alloy actuators for haptic wearables: A reviewMaterials & Design10.1016/j.matdes.2023.112264233(112264)Online publication date: Sep-2023
https://doi.org/10.1016/j.matdes.2023.112264
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten