Skip to main content
Springer Nature Link
Log in

Analyzing the Interplay of Visual and Auditory Stimuli on Human Physiology and Stress Levels in Virtual Environment

  • Conference paper
  • First Online:
HCI International 2024 – Late Breaking Papers (HCII 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15376))

Included in the following conference series:

  • 248 Accesses

Abstract

This investigation extends prior research on the impact of audio stimuli in Virtual Reality (VR) environments, with an enhanced focus on understanding the relationship between audio stimuli and the level of immersion, as indicated by stress levels. Using the State-Trait Anxiety Inventory (STAI) for psychological assessment and heart rate variability for physiological responses, this study examines the interplay of visual and auditory stimuli across four diverse VR scenarios. These scenarios include combinations of city and forest environments paired alternately with matching and contrasting auditory cues. Correlation analysis was employed to determine how audio stimuli influence the level of immersion, reflected through variations in heart rate and STAI-derived stress indicators. Participants’ responses were continuously monitored for real-time physiological effects, while stress assessments were conducted after exposure to each scenario. This methodical approach aims to uncover significant sensory interactions that affect immersion in VR, potentially guiding the design and application of VR technologies across various disciplines.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Apostolou, K., Liarokapis, F.: A systematic review: the role of multisensory feedback in virtual reality. In: 2nd International Conference on Intelligent Reality (ICIR), pp. 39–42. IEEE (2022)

    Google Scholar 

  2. Basharat, A., et al.: Virtual reality as a tool to explore multisensory processing before and after engagement in physical activity. Front. Aging Neurosci. 15, 1207651 (2023)

    Article  Google Scholar 

  3. Canales, R., Jörg, S.: Performance is not everything: audio feedback preferred over visual feedback for grasping task in virtual reality. In: Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, pp. 1–6 (2020)

    Google Scholar 

  4. Chen, H., Dey, A., Billinghurst, M., Linderman, R.W.: Exploring the design space for multi-sensory heart rate feedback in immersive virtual reality. In: Proceedings of the 29th Australian conference on computer-human interaction, pp. 108–116 (2017)

    Google Scholar 

  5. Chen, T., Wu, Y.S., Zhu, K.: Investigating different modalities of directional cues for multi-task visual-searching scenario in virtual reality. In: Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology, pp. 1–5 (2018)

    Google Scholar 

  6. Cooper, N., Milella, F., Pinto, C., Cant, I., White, M., Meyer, G.: The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment. PLoS ONE 13(2), e0191846 (2018)

    Article  Google Scholar 

  7. Grassini, S., Laumann, K., de Martin Topranin, V., Thorp, S.: Evaluating the effect of multi-sensory stimulations on simulator sickness and sense of presence during HMD-mediated VR experience. Ergonomics 64(12), 1532–1542 (2021)

    Article  Google Scholar 

  8. Halbig, A., Latoschik, M.E.: A systematic review of physiological measurements, factors, methods, and applications in virtual reality. Front. Virtual Reality 2, 694567 (2021)

    Google Scholar 

  9. Hirao, Y., Kawai, T.: Augmented cross-modality: Translating the physiological responses, knowledge, and impression to audio-visual information in virtual reality. Electron. Imaging 31, 1–8 (2018)

    Google Scholar 

  10. Kamińska, D., Smólka, K., Zwoliński, G.: Detection of mental stress through EEG signal in virtual reality environment. Electronics 10(22), 2840 (2018)

    Article  Google Scholar 

  11. Kim, H., et al.: Effect of virtual reality on stress reduction and change of physiological parameters including heart rate variability in people with high stress: an open randomized crossover trial. Front. Psychiatry 12, 614539 (2021)

    Google Scholar 

  12. Kim, J., Luu, W., Palmisano, S.: Multisensory integration and the experience of scene instability, presence and cybersickness in virtual environments. Comput. Hum. Behav. 113, 106484 (2020)

    Google Scholar 

  13. Liu, F., Kang, J.: Relationship between street scale and subjective assessment of audio-visual environment comfort based on 3D virtual reality and dual-channel acoustic tests. Build. Environ. 129, 35–45 (2018)

    Article  Google Scholar 

  14. Marchiori, E., Niforatos, E., Preto, L.: Analysis of users’ heart rate data and self-reported perceptions to understand effective virtual reality characteristics. Inf. Technol. Tourism 18, 133–155 (2018)

    Google Scholar 

  15. Melo, M., Gonçalves, G., Monteiro, P., Coelho, H., Vasconcelos-Raposo, J., Bessa, M.: Do multisensory stimuli benefit the virtual reality experience? A systematic review. IEEE Trans. Visual Comput. Graphics 28(2), 1428–1442 (2020)

    Article  Google Scholar 

  16. Oing, T., Prescott, J.: Implementations of virtual reality for anxiety-related disorders: systematic review. JMIR serious games 6(4), e10965 (2018)

    Article  Google Scholar 

  17. Ranasinghe, N., Jain, P., Tolley, D., Karwita Tailan, S., Yen, C.C., Do, E.Y.L.: Exploring the use of olfactory stimuli towards reducing visually induced motion sickness in virtual reality. In: Proceedings of the 2020 ACM Symposium on Spatial User Interaction, pp. 1–9 (2020)

    Google Scholar 

  18. Riches, S., Azevedo, L., Bird, L., Pisani, S., Valmaggia, L.: Virtual reality relaxation for the general population: a systematic review. Soc. Psychiatry Psychiatr. Epidemiol. 56, 1707–1727 (2021)

    Article  Google Scholar 

  19. Salgado, D.P., Martins, F.R., Rodrigues, T.B., Keighrey, C., Flynn, R., Naves, E.L.M., Murray, N.: A QoE assessment method based on EDA, heart rate and EEG of a virtual reality assistive technology system. In: Proceedings of the 9th ACM Multimedia Systems Conference, pp. 517–520 (2018)

    Google Scholar 

  20. Šalkevicius, J., Damaševičius, R., Maskeliunas, R., Laukienė, I.: Anxiety level recognition for virtual reality therapy system using physiological signals. Electronics 8(9), 1039 (2019)

    Article  Google Scholar 

  21. Sanchez, G.M.E., Van Renterghem, T., Sun, K., De Coensel, B., Botteldooren, D.: Using Virtual Reality for assessing the role of noise in the audio-visual design of an urban public space. Landsc. Urban Plan. 167, 98–107 (2017)

    Article  Google Scholar 

  22. Tabbaa, L., et al.: Vreed: Virtual reality emotion recognition dataset using eye tracking and physiological measures. Proc. ACM Interact. Mobile, Wearable Ubiquit. Technol. 5(4), 1–20 (2021)

    Article  Google Scholar 

  23. Wu, F., Thomas, J., Chinnola, S., Rosenberg, E.S.: Comparison of audio and visual cues to support remote guidance in immersive environments. In: ICAT-EGVE, pp. 121–130 (2020)

    Google Scholar 

  24. Wang, X., Shi, Y., Zhang, B., Chiang, Y.: The influence of forest resting environments on stress using virtual reality. Int. J. Environ. Res. Public Health 16(18), 3263 (2019)

    Article  Google Scholar 

  25. Zeng, N., Pope, Z., Lee, J.E., Gao, Z.: Virtual reality exercise for anxiety and depression: a preliminary review of current research in an emerging field. J. Clin. Med. 7(3), 42 (2018)

    Article  Google Scholar 

  26. Shattuck, D.W.: Multiuser virtual reality environment for visualising neuroimaging data. Healthc. Technol. Lett. 5(5), 183–188 (2018)

    Article  Google Scholar 

  27. Moosavi, M.S., et al.: The influence of audio sensory input on the anxiety level of the VR Users. In: International Conference on Future Trends in Smart Communities (ICFTSC), pp. 237–242. IEEE (2022)

    Google Scholar 

  28. Spielberger, C.D., Gonzalez-Reigosa, F., Martinez-Urrutia, A., Natalicio, L.F.S., Natalicio, D.S.: The state-trait anxiety inventory. Rev. Interamericana de Psicologia/Interamerican J. Psychol. 5, 3–4 (1971)

    Google Scholar 

Download references

Acknowledgments

With profound gratitude, the authors gratefully acknowledge the support received from the France Embassy in Malaysia. We also acknowledge the institutional support provided by Asia Pacific University of Technology and Innovation, Kuala Lumpur, Malaysia, in collaboration with the Arts et Metiers Institute of Technology LISPEN, HESAM Université, Chalon-Sur-Saone, France. Their support and collaboration were instrumental in facilitating this research endeavour.

Author information

Authors and Affiliations

  1. School of Computing, Asia Pacific University of Technology and Innovation, Kuala Lumpur, Malaysia

    Yik Junn Kuan & Angelina Seow Voon Yee

  2. Arts et Metiers Institute of Technology, LISPEN, HESAM Université, 71100, Chalon-Sur-Saone, France

    Mahdiyeh Sadat Moosavi & Frédéric Merienne

  3. Universite de Bourgogne, LISPEN, UBFC, 71100, Chalon-Sur-Saone, France

    Christophe Guillet

Authors
  1. Yik Junn Kuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahdiyeh Sadat Moosavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christophe Guillet
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Angelina Seow Voon Yee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Frédéric Merienne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Mahdiyeh Sadat Moosavi , Christophe Guillet , Angelina Seow Voon Yee or Frédéric Merienne .

Editor information

Editors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Vincent G. Duffy

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kuan, Y.J., Moosavi, M.S., Guillet, C., Yee, A.S.V., Merienne, F. (2025). Analyzing the Interplay of Visual and Auditory Stimuli on Human Physiology and Stress Levels in Virtual Environment. In: Duffy, V.G. (eds) HCI International 2024 – Late Breaking Papers. HCII 2024. Lecture Notes in Computer Science, vol 15376. Springer, Cham. https://doi.org/10.1007/978-3-031-76809-5_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-76809-5_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-76808-8

  • Online ISBN: 978-3-031-76809-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics