Skip to main content
SpringerLink
Log in
Book cover

International Conference on Virtual Reality and Augmented Reality

EuroVR 2020: Virtual Reality and Augmented Reality pp 83–95Cite as

Psychophysical Effects of Experiencing Burning Hands in Augmented Reality

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12499))

Abstract

Can interactive Augmented Reality (AR) experiences induce involuntary sensations through additional modalities? In this paper we report on our AR experience that enables users to see and hear their own hands burning while looking through a Video See-Through Head-Mounted Display (VST-HMD). In an exploratory study (n = 12, within-subject design), we investigated whether this will lead to an involuntary heat sensation based on visual and auditory stimuli. A think-aloud-protocol and an AR presence questionnaire indicated that six out of twelve participants experienced an involuntary heat sensation on their hands. Despite no significant change of perceived anxiety, we found a significant increase in skin conductance during the experiment for all participants; participants who reported an involuntary heat sensation had higher skin conductance responses than participants who did not report a heat sensation. Our results support our initial hypothesis as we found evidence of cross-modal audiovisual-to-thermal transfers. This is an example of virtual synaesthesia, a sensation occurring when single-modal (or multi-modal) stimulus sets off the simultaneous sensation over other senses—involuntarily and automatically. We believe that our results contribute to the scientific understanding of AR induced synaesthesia as well as inform practical applications.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Graph. Appl. 21(6) (2001). https://doi.org/10.1109/38.963459

  2. Biocca, F., Jin, K., Choi, Y.: Visual touch in virtual environments: an exploratory study of presence, multimodal interfaces, and cross-modal sensory illusions. Presence: Teleoperators Virtual Environ. 10(3), 247–265 (2001). https://doi.org/10.1162/105474601300343595

  3. Braithwaite, J., Watson, D., Robert, J., Mickey, R.: A guide for analysing electrodermal activity (EDA) & skin conductance responses (SCRs) for psychological experiments. Psychophysiology 49, 1017–1034 (2013).

    Google Scholar 

  4. Chadwick, J.N., James, D.L.: Animating fire with sound. ACM Trans. Graph. (TOG) 30(4), 1–8 (2011)

    Article  Google Scholar 

  5. Coelho, C., Tichon, J., Hine, T.J., Wallis, G., Riva, G.: Media presence and inner presence: the sense of presence in virtual reality technologies. Emerging Communication: Studies in New Technologies and Practices in Communication, vol. 9, pp. 25–45 (2012)

    Google Scholar 

  6. Collins, J., Regenbrecht, H., Langlotz, T., Can, Y.S., Ersoy, C., Butson, R.: Measuring cognitive load and insight: a methodology exemplified in a virtual reality learning context. In: Proceedings - 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2019, Beijing, China, pp. 351–362. IEEE (2019). https://doi.org/10.1109/ISMAR.2019.00033

  7. Cytowic, R.E.: Synesthesia: A Union of the Senses. MIT Press, London (2002)

    Book  Google Scholar 

  8. Davis, E.T., Scott, K., Pair, J., Hodges, L.F., Oliverio, J.: Can audio enhance visual perception and performance in a virtual environment? In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 43, no. 22, pp. 1197–1201 (1999). https://doi.org/10.1177/154193129904302206

  9. Dawson, M.E., Schell, A.M., Filion, D.L.: The electrodermal system. In: Handbook of Psychophysiology, Fourth Edition, pp. 217–243. Cambridge University Press, New York (2017). https://doi.org/10.1017/9781107415782.010

  10. Dewez, D., et al.: Influence of personality traits and body awareness on the sense of embodiment in virtual reality. In: Proceedings - 2019 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2019, Beijing, China, pp. 123–134. IEEE (2019). https://doi.org/10.1109/ISMAR.2019.00-12

  11. Durgin, F.H., Evans, L., Dunphy, N., Klostermann, S., Simmons, K.: Rubber hands feel the touch of light. Psychol. Sci. 18(2), 152–157 (2007). https://doi.org/10.1111/j.1467-9280.2007.01865.x

    Article  Google Scholar 

  12. Felnhofer, A., Kothgassner, O.D., Hetterle, T., Beutl, L., Hlavacs, H., Kryspin-Exner, I.: Afraid to be there? Evaluating the relation between presence, self-reported anxiety, and heart rate in a virtual public speaking task. Cyberpsychol. Behav. Soc. Netw. 17(5), 310–316 (2014). https://doi.org/10.1089/cyber.2013.0472

    Article  Google Scholar 

  13. Gallagher, E.J.: No proof of a difference is not equivalent to proof of no difference. J. Emerg. Med. 12(4), 525–527 (1994)

    Article  Google Scholar 

  14. Gandy, M., et al.: Experiences with an AR evaluation test bed: presence, performance, and physiological measurement. In: 9th IEEE International Symposium on Mixed and Augmented Reality 2010: Science and Technology, ISMAR 2010 - Proceedings, Seoul, Korea, pp. 127–136. IEEE (2010). https://doi.org/10.1109/ISMAR.2010.5643560

  15. Greco, A., Valenza, G., Lanata, A., Scilingo, E.P., Citi, L.: CvxEDA: a convex optimization approach to electrodermal activity processing. IEEE Trans. Biomed. Eng. 63(4), 797–804 (2016). https://doi.org/10.1109/TBME.2015.2474131

    Article  Google Scholar 

  16. Guerreiro, J., Martins, R., da Silva, H., Lourenco, A., Fred, A.: BITalino: a multimodal platform for physiological computing. In: ICINCO 2013 - Proceedings of the 10th International Conference on Informatics in Control, Automation and Robotics, Reykjavík, Iceland, vol. 1, pp. 500–506. INSTICC (2013)

    Google Scholar 

  17. Hoffman, H.G., et al.: Modulation of thermal pain-related brain activity with virtual reality: evidence from fMRI. NeuroReport 15(8), 1245–1248 (2004). https://doi.org/10.1097/01.wnr.0000127826.73576.91

    Article  Google Scholar 

  18. HTC: HTC Vive pro (2020). https://www.vive.com/hk/product/vive-pro/

  19. Iacobucci, D., Posavac, S.S., Kardes, F.R., Schneider, M.J., Popovich, D.L.: Toward a more nuanced understanding of the statistical properties of a median split. J. Consum. Psychol. 25(4), 652–665 (2015). https://doi.org/10.1016/j.jcps.2014.12.002

    Article  Google Scholar 

  20. Jacobs, S.C., et al.: Use of skin conductance changes during mental stress testing as an index of autonomic arousal in cardiovascular research. Am. Heart J. 128(6 Part 1), 1170–1177 (1994). https://doi.org/10.1016/0002-8703(94)90748-X

  21. JASP Team: JASP (Version 0.12.2). https://jasp-stats.org/

  22. Juan, M.C., Alcañiz, M., Monserrat, C., Botella, C., Baños, R.M., Guerrero, B.: Using augmented reality to treat phobias. IEEE Comput. Graphics Appl. 25(6), 31–37 (2005). https://doi.org/10.1109/MCG.2005.143

    Article  Google Scholar 

  23. Krijn, M., Emmelkamp, P.M., Olafsson, R.P., Biemond, R.: Virtual reality exposure therapy of anxiety disorders: a review (2004). https://doi.org/10.1016/j.cpr.2004.04.001

  24. Lecuyer, A., Coquillart, S., Kheddar, A., Richard, P., Coiffet, P.: Pseudo-haptic feedback: can isometric input devices simulate force feedback? In: Proceedings - Virtual Reality Annual International Symposium, New Brunswick, NJ, US, pp. 83–90. IEEE (2000). https://doi.org/10.1109/vr.2000.840369

  25. Makowski, D.: NeuroKit: A Python Toolbox for Statistics and Neurophysiological Signal Processing (EEG, EDA, ECG, EMG...) (2016)

    Google Scholar 

  26. Martens, M.A., Antley, A., Freeman, D., Slater, M., Harrison, P.J., Tunbridge, E.M.: It feels real: physiological responses to a stressful virtual reality environment and its impact on working memory. J. Psychopharmacol. 33(10), 1264–1273 (2019). https://doi.org/10.1177/0269881119860156

    Article  Google Scholar 

  27. Nambu, A., Narumi, T., Nishimura, K., Tanikawa, T., Hirose, M.: Visual-olfactory display using olfactory sensory map. In: Proceedings - IEEE Virtual Reality, Waltham, Massachusetts, USA, pp. 39–42. IEEE (2010). https://doi.org/10.1109/VR.2010.5444817

  28. Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., Hirose, M.: Augmented reality flavors: gustatory display based on Edible Marker and cross-modal interaction. In: Conference on Human Factors in Computing Systems - Proceedings, Vancouver, Canada, pp. 93–102. ACM SIGCHI (2011). https://doi.org/10.1145/1978942.1978957

  29. Pinel, J.P.J.: Biopsychology of Emotion. Stress and Health (2011)

    Google Scholar 

  30. Pusch, A., Martin, O., Coquillart, S.: HEMP-hand-displacement-based pseudo-haptics: a study of a force field application and a behavioural analysis. Int. J. Hum. Comput. Stud. 67(3), 256–268 (2009). https://doi.org/10.1016/j.ijhcs.2008.09.015

    Article  Google Scholar 

  31. Rogowska, A.: Categorization of synaesthesia. Rev. General Psychol. 15(3), 213–227 (2011). https://doi.org/10.1037/a0024078

    Article  Google Scholar 

  32. Saha, D.P., Knapp, R.B., Martin, T.L.: Affective feedback in a virtual reality based intelligent supermarket. In: UbiComp/ISWC 2017 - Adjunct Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, Hawai’i, USA, pp. 646–653. ACM (2017). https://doi.org/10.1145/3123024.3124426

  33. Samana, R., Wallach, H.S., Safir, M.P.: The impact of personality traits on the experience of presence. In: 2009 Virtual Rehabilitation International Conference, VR 2009, Haifa, Israel, pp. 1–7. IEEE (2009). https://doi.org/10.1109/ICVR.2009.5174197

  34. Sanchez-Vives, M.V., Slater, M.: From presence to consciousness through virtual reality (2005). https://doi.org/10.1038/nrn1651

  35. Sandor, C., et al.: Breaking the barriers to true augmented reality. CoRR abs/1512.05471 (2015). http://arxiv.org/abs/1512.05471

  36. Slater, M., Steed, A.: A virtual presence counter. Presence: Teleoperators Virtual Environ. 9(5), 413–434 (2000). https://doi.org/10.1162/105474600566925

  37. Spielberger, C.D.: State-Trait Anxiety Inventory (2010). https://doi.org/10.1002/9780470479216.corpsy0943

  38. Steptoe, W., Steed, A., Slater, M.: Human tails: ownership and control of extended humanoid avatars. IEEE Trans. Vis. Comput. Graph. 19(4), 583–590 (2013). https://doi.org/10.1109/TVCG.2013.32. https://doi.org/10.2312/egve.20151320

  39. Tomaka, J., Blascovich, J., Kelsey, R.M., Leitten, C.L.: Subjective, physiological, and behavioral effects of threat and challenge appraisal. J. Pers. Soc. Psychol. 65(2), 248–260 (1993). https://doi.org/10.1037/0022-3514.65.2.248

    Article  Google Scholar 

  40. Tomaka, J., Palacios, R., Schneider, K.T., Colotla, M., Concha, J.B., Herrald, M.M.: Assertiveness predicts threat and challenge reactions to potential stress among women. J. Pers. Soc. Psychol. 76(6), 1008–1021 (1999). https://doi.org/10.1037/0022-3514.76.6.1008

    Article  Google Scholar 

  41. Weibel, D., Wissmath, B., Mast, F.W.: Immersion in mediated environments: the role of personality traits. Cyberpsychol. Behav. Soc. Netwo. 13(3), 251–256 (2010). https://doi.org/10.1089/cyber.2009.0171

    Article  Google Scholar 

  42. Weir, P., et al.: Burnar: involuntary heat sensations in augmented reality. In: Proceedings - IEEE Virtual Reality, Orlando, Florida, pp. 43–46. IEEE (2013). https://doi.org/10.1109/VR.2013.6549357

  43. Weir, P.C.: Effects of Augmented Reality on Bodily Perception, October 2013

    Google Scholar 

  44. Yeh, S.C., Li, Y.Y., Zhou, C., Chiu, P.H., Chen, J.W.: Effects of virtual reality and augmented reality on induced anxiety. IEEE Trans. Neural Syst. Rehabil. Eng. 26(7), 1345–1352 (2018)

    Article  Google Scholar 

  45. Youngblut, C.: What a decade of experiments reveals about factors that influence the sense of presence. Technical report, Institute for Defense Analyses Alexandria VA (2006)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. City University of Hong Kong, Kowloon, Hong Kong

    Daniel Eckhoff, Alvaro Cassinelli & Christian Sandor

  2. University of Oldenburg, Oldenburg, Germany

    Tuo Liu

Authors
  1. Daniel Eckhoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alvaro Cassinelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tuo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Sandor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Eckhoff .

Editor information

Editors and Affiliations

  1. VENISE team, LIMSI-CNRS, Paris-Saclay University, Orsay, France

    Patrick Bourdot

  2. University of Minnesota, Minneapolis, MN, USA

    Victoria Interrante

  3. University of North Carolina at Greensboro, Greensboro, NC, USA

    Regis Kopper

  4. University of Rennes 2, Rennes, France

    Anne-Hélène Olivier

  5. Department of Information and Computer Science, Keio University, Yokohama, Japan

    Hideo Saito

  6. University of Bremen, Bremen, Bremen, Germany

    Gabriel Zachmann

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Eckhoff, D., Cassinelli, A., Liu, T., Sandor, C. (2020). Psychophysical Effects of Experiencing Burning Hands in Augmented Reality. In: Bourdot, P., Interrante, V., Kopper, R., Olivier, AH., Saito, H., Zachmann, G. (eds) Virtual Reality and Augmented Reality. EuroVR 2020. Lecture Notes in Computer Science(), vol 12499. Springer, Cham. https://doi.org/10.1007/978-3-030-62655-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62655-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62654-9

  • Online ISBN: 978-3-030-62655-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation