skip to main content
10.1145/3488162.3488228acmotherconferencesArticle/Chapter ViewAbstractPublication PagessiggraphConference Proceedingsconference-collections
research-article

Virtual Reality Assessment and Customization Using Physiological Measures: A Literature Analysis

Published: 03 January 2022 Publication History

Abstract

Understanding the user experience requires gathering self-reported and physiological information. In Virtual Reality (VR), heart rate, skin responses, gaze, brain, and muscle activity can provide a better understanding of how users process perceptual cues and respond to them. While the use of physiological measures is typically done for the assessment and improvement of the user experience, recent trends have shown their application for customizing VR experiences to the unique responses of the users. The purpose of this literature review is to analyze the use of physiological measures in VR customization to identify the state of the art, gaps, and opportunities. A total of 25 articles from indexed journals found in Scopus, PubMed, Web of Science, and IEEE, in addition to top tier conferences such as IEEE VR and CHI within the last five years were reviewed. The review indicates that while assessment continues to be a main field of application of physiological measures, the recent availability of consumer-level VR and the growth of ubiquitous physiological sensors are enabling their use as user input devices for customized and adaptive user experiences.

References

[1]
Isayas Berhe Adhanom, Nathan Navarro Griffin, Paul MacNeilage, and Eelke Folmer. 2020. The Effect of a Foveated Field-of-view Restrictor on VR Sickness. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). Institute of Electrical and Electronics Engineers (IEEE), Atlanta, GA, USA, 645–652. https://doi.org/10.1109/vr46266.2020.00087
[2]
Mike Ambinder. 2009. Valve’s approach to playtesting: The application of empiricism. http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Valve's+approach+to+playtesting:+The+application+of+empiricism#0
[3]
Peter Arnold. 2017. You Better Eat to Survive! Exploring edible interactions in a virtual reality game. Conference on Human Factors in Computing Systems - Proceedings Part F127655 (may 2017), 206–209. https://doi.org/10.1145/3027063.3048408
[4]
Mehmet Ilker Berkman and Ecehan Akan. 2019. Presence and Immersion in Virtual Reality. In Encyclopedia of Computer Graphics and Games. Springer, Cham, Cham, 1–10. https://doi.org/10.1007/978-3-319-08234-9_162-1
[5]
Sergi Bermúdez i Badia, Luis Velez Quintero, Mónica S. Cameirão, Alice Chirico, Stefano Triberti, Pietro Cipresso, and Andrea Gaggioli. 2019. Toward Emotionally Adaptive Virtual Reality for Mental Health Applications. IEEE Journal of Biomedical and Health Informatics 23, 5(2019), 1877–1887. https://doi.org/10.1109/JBHI.2018.2878846
[6]
Nigel Bevan, Jurek Kirakowski, and Jonathan Maissel. 1991. What is Usability?Proceedings of the 4th International Conference on HC 1, 1(1991), 527–531.
[7]
Dayi Bian, Joshua Wade, Amy Swanson, Amy Weitlauf, Zachary Warren, and Nilanjan Sarkar. 2019. Design of a physiology-based adaptive virtual reality driving platform for individuals with ASD. ACM Transactions on Accessible Computing 12, 1 (feb 2019), 1–24. https://doi.org/10.1145/3301498
[8]
Rebecca L. Charles and Jim Nixon. 2019. Measuring mental workload using physiological measures: A systematic review., 221–232 pages. https://doi.org/10.1016/j.apergo.2018.08.028
[9]
Yang Chen, Arya Xinran Huang, Ilona Faber, Guido Makransky, and Federico J.A. Perez-Cueto. 2020. Assessing the influence of visual-taste congruency on perceived sweetness and product liking in immersive VR. Foods 9, 4 (apr 2020), 465. https://doi.org/10.3390/foods9040465
[10]
Adyasha Dash and Uttama Lahiri. 2020. Design of Virtual Reality-Enabled Surface Electromyogram-Triggered Grip Exercise Platform. IEEE Transactions on Neural Systems and Rehabilitation Engineering 28, 2 (feb 2020), 444–452. https://doi.org/10.1109/TNSRE.2019.2959449
[11]
Victor Delvigne, Hazem Wannous, Thierry Dutoit, Laurence Ris, and Jean Philippe Vandeborre. 2021. PhyDAA: Physiological Dataset Assessing Attention. IEEE Transactions on Circuits and Systems for Video Technology 1 (2021), 1–1. https://doi.org/10.1109/TCSVT.2021.3061719
[12]
Natalia Duzmanska, Pawel Strojny, and Agnieszka Strojny. 2018. Can simulator sickness be avoided? A review on temporal aspects of simulator sickness. https://doi.org/10.3389/fpsyg.2018.02132
[13]
Ziqi Fan, Kaitlyn Brown, Stephanie Nistor, Karishma Seepaul, Kody Wood, Alvaro Uribe-Quevedo, Sharman Perera, Edward Waller, and Shawn Lowe. 2020. Use of Virtual Reality Technology for CANDU 6 Reactor Fuel Channel Operation Training. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12517 LNCS. Springer, Cham, Oshawa, Canada, 91–101. https://doi.org/10.1007/978-3-030-63464-3_9
[14]
Mikhail Fominykh, Ekaterina Prasolova-Førland, Tore C. Stiles, Anne Berit Krogh, and Mattias Linde. 2018. Conceptual framework for therapeutic training with biofeedback in virtual reality: First evaluation of a relaxation simulator. Journal of Interactive Learning Research 29, 1 (2018), 51–75.
[15]
Jože Guna, Gregor Geršak, Iztok Humar, Jeungeun Song, Janko Drnovšek, and Matevž Pogačnik. 2019. Influence of video content type on users’ virtual reality sickness perception and physiological response. Future Generation Computer Systems 91 (2019), 263–276. https://doi.org/10.1016/j.future.2018.08.049
[16]
Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52, C (jan 1988), 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9
[17]
Razeen Hussain, Manuela Chessa, and Fabio Solari. 2021. Mitigating cybersickness in virtual reality systems through foveated depth-of-field blur. Sensors 21, 12 (jun 2021), 4006. https://doi.org/10.3390/s21124006
[18]
Kodai Ito and Michiko Ohkura. 2017. Evaluation of ’dokidoki feelings’ for a VR system using ECGs with comparison between genders. In Proceedings of 2017 International Conference on Biometrics and Kansei Engineering, ICBAKE 2017. Institute of Electrical and Electronics Engineers Inc., Kyoto, 110–114. https://doi.org/10.1109/ICBAKE.2017.8090623
[19]
M. P. Jacob Habgood, David Moore, David Wilson, and Sergio Alapont. 2018. Rapid, Continuous Movement between Nodes as an Accessible Virtual Reality Locomotion Technique. In 25th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2018 - Proceedings. Institute of Electrical and Electronics Engineers Inc., Tuebingen/Reutlingen, Germany, 371–378. https://doi.org/10.1109/VR.2018.8446130
[20]
Charlene Jennett, Anna L. Cox, Paul Cairns, Samira Dhoparee, Andrew Epps, Tim Tijs, and Alison Walton. 2008. Measuring and defining the experience of immersion in games. International Journal of Human Computer Studies 66, 9 (sep 2008), 641–661. https://doi.org/10.1016/j.ijhcs.2008.04.004
[21]
David Johnson, Daniela Damian, and George Tzanetakis. 2020. Evaluating the effectiveness of mixed reality music instrument learning with the theremin. Virtual Reality 24, 2 (jun 2020), 303–317. https://doi.org/10.1007/s10055-019-00388-8
[22]
Sungchul Jung, Richard Li, Ryan McKee, Mary C. Whitton, and Robert W. Lindeman. 2021. Floor-vibration VR: Mitigating Cybersickness Using Whole-body Tactile Stimuli in Highly Realistic Vehicle Driving Experiences. IEEE Transactions on Visualization and Computer Graphics 27, 5 (may 2021), 2669–2680. https://doi.org/10.1109/TVCG.2021.3067773
[23]
Michal Kafri, Patrice L Weiss, Gabriel Zeilig, Moshe Bondi, Ilanit Baum-Cohen, and Rachel Kizony. 2021. Performance in complex life situations: effects of age, cognition, and walking speed in virtual versus real life environments. Journal of NeuroEngineering and Rehabilitation 18, 1(2021), 1–9.
[24]
Michal Kafri, Patrice L. Weiss, Gabriel Zeilig, Moshe Bondi, Ilanit Baum-Cohen, and Rachel Kizony. 2021. Performance in complex life situations: effects of age, cognition, and walking speed in virtual versus real life environments. Journal of NeuroEngineering and Rehabilitation 18, 1 (feb 2021), NA–NA. https://doi.org/10.1186/s12984-021-00830-6
[25]
Saleh Kalantari, James D Rounds, Julia Kan, Vidushi Tripathi, and Jesus G Cruz-Garza. 2021. Comparing physiological responses during cognitive tests in virtual environments vs. in identical real-world environments. Scientific Reports 11, 1 (2021), 1–14.
[26]
Saleh Kalantari, James D. Rounds, Julia Kan, Vidushi Tripathi, and Jesus G. Cruz-Garza. 2021. Comparing physiological responses during cognitive tests in virtual environments vs. in identical real-world environments. Scientific Reports 11, 1 (may 2021), 1–14. https://doi.org/10.1038/s41598-021-89297-y
[27]
Dorota Kamiska, Krzysztof Smóka, Grzegorz Zwoliski, S. Awomir Wiak, Dorota Merecz-Kot, and Gholamreza Anbarjafari. 2020. Stress Reduction Using Bilateral Stimulation in Virtual Reality. IEEE Access 8(2020), 200351–200366. https://doi.org/10.1109/ACCESS.2020.3035540
[28]
Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (1993), 203–220. https://doi.org/10.1207/s15327108ijap0303_3
[29]
Si Jung Kim, Teemu H. Laine, and Hae Jung Suk. 2021. Presence Effects in Virtual Reality Based on User Characteristics: Attention, Enjoyment, and Memory. Electronics 10, 9 (apr 2021), 1051. https://doi.org/10.3390/electronics10091051
[30]
Panos Kostakos, Paula Alavesa, Mikko Korkiakoski, Mario Monteiro Marques, Victor Lobo, and Filipe Duarte. 2021. Wired to Exit: Exploring the Effects of Wayfinding Affordances in Underground Facilities Using Virtual Reality. Simulation and Gaming 52, 2 (aug 2021), 107–131. https://doi.org/10.1177/1046878120944567
[31]
Jacob Kritikos, Georgios Alevizopoulos, and Dimitris Koutsouris. 2021. Personalized virtual reality human-computer interaction for psychiatric and neurological illnesses: a dynamically adaptive virtual reality environment that changes according to real-time feedback from electrophysiological signal responses. Frontiers in Human Neuroscience 15 (2021), 45.
[32]
S. J. Lackey, J. N. Salcedo, J. L. Szalma, and P. A. Hancock. 2016. The stress and workload of virtual reality training: the effects of presence, immersion and flow. Ergonomics 59, 8 (aug 2016), 1060–1072. https://doi.org/10.1080/00140139.2015.1122234
[33]
Jeroen S. Lemmens, Monika Simon, and Sindy R. Sumter. 2021. Fear and loathing in VR: the emotional and physiological effects of immersive games. Virtual Reality 2021 1 (jul 2021), 1–12. https://doi.org/10.1007/S10055-021-00555-W
[34]
James R. Lewis. 2018. The System Usability Scale: Past, Present, and Future. International Journal of Human-Computer Interaction 34, 7 (jul 2018), 577–590. https://doi.org/10.1080/10447318.2018.1455307
[35]
Jing Lin, Lijun Cao, and Nan Li. 2019. Assessing the influence of repeated exposures and mental stress on human wayfinding performance in indoor environments using virtual reality technology. Advanced Engineering Informatics 39 (jan 2019), 53–61. https://doi.org/10.1016/j.aei.2018.11.007
[36]
Michael Meehan, Sharif Razzaque, Mary C. Whitton, and Frederick P. Brooks. 2003. Effect of latency on presence in stressful virtual environments. In Proceedings - IEEE Virtual Reality, Vol. 2003-Janua. IEEE Computer Society, Los Angeles, CA, USA, 141–148. https://doi.org/10.1109/VR.2003.1191132
[37]
Christian Moro, Zane Štromberga, Athanasios Raikos, and Allan Stirling. 2017. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anatomical Sciences Education 10, 6 (2017), 549–559. https://doi.org/10.1002/ase.1696
[38]
Matthias Neges, Stefan Adwernat, and Michael Abramovici. 2018. Augmented Virtuality for maintenance training simulation under various stress conditions. In Procedia Manufacturing, Vol. 19. Elsevier, Bremen, Germany, 171–178. https://doi.org/10.1016/j.promfg.2018.01.024
[39]
Tatiana Ortegon-Sarmiento, Lina Penuela, and Alvaro Uribe-Quevedo. 2020. Low Back Pain Attenuation Employing Virtual Reality Physiotherapy. In Proceedings - 2020 22nd Symposium on Virtual and Augmented Reality, SVR 2020. Institute of Electrical and Electronics Engineers Inc., Porto de Galinhas, Brazil, 169–173. https://doi.org/10.1109/SVR51698.2020.00037
[40]
Seiya Otsuka, Kanami Kurosaki, and Mitsuhiro Ogawa. 2017. Physiological measurements on a gaming virtual reality headset using photoplethysmography: A preliminary attempt at incorporating physiological measurement with gaming. IEEE Region 10 Annual International Conference, Proceedings/TENCON 2017-Decem (dec 2017), 1251–1256. https://doi.org/10.1109/TENCON.2017.8228049
[41]
Federica Pallavicini, Alessandro Pepe, and Maria Eleonora Minissi. 2019. Gaming in Virtual Reality: What Changes in Terms of Usability, Emotional Response and Sense of Presence Compared to Non-Immersive Video Games?Simulation and Gaming 50, 2 (2019), 136–159. https://doi.org/10.1177/1046878119831420
[42]
Steven M. Peterson, Emily Furuichi, and Daniel P. Ferris. 2018. Effects of virtual reality high heights exposure during beam-walking on physiological stress and cognitive loading. PLoS ONE 13, 7 (jul 2018), e0200306. https://doi.org/10.1371/journal.pone.0200306
[43]
Mores Prachyabrued, Disathon Wattanadhirach, Richard B. Dudrow, Nat Krairojananan, and Pusit Fuengfoo. 2019. Toward virtual stress inoculation training of prehospital healthcare personnel: A stress-inducing environment design and investigation of an emotional connection factor. In 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., Osaka, Japan, 671–679. https://doi.org/10.1109/VR.2019.8797705
[44]
Marco Recenti, Carlo Ricciardi, Romain Aubonnet, Ilaria Picone, Deborah Jacob, Halldór Á.R. Svansson, Sólveig Agnarsdóttir, Gunnar H. Karlsson, Valdís Baeringsdóttir, Hannes Petersen, and Paolo Gargiulo. 2021. Toward Predicting Motion Sickness Using Virtual Reality and a Moving Platform Assessing Brain, Muscles, and Heart Signals. Frontiers in Bioengineering and Biotechnology 9 (apr 2021), 132. https://doi.org/10.3389/fbioe.2021.635661
[45]
Jonathan Saunders, Steffi Davey, Petra Saskia Bayerl, and Philipp Lohrmann. 2019. Validating virtual reality as an effective training medium in the security domain. In 26th IEEE Conference on Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings. Institute of Electrical and Electronics Engineers Inc., Osaka, Japan, 1908–1911. https://doi.org/10.1109/VR.2019.8798371
[46]
Peter Schulz, Dmitry Alexandrovsky, Felix Putze, Rainer Malaka, and Johannes Schöning. 2019. The Role of Physical Props in VR Climbing Environments. In CHI ’19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, US, 1–13. https://doi.org/10.1145/3290605.3300413
[47]
Andrej Somrak, Matevž Pogačnik, and Jože Guna. 2021. Suitability and comparison of questionnaires assessing virtual reality-induced symptoms and effects and user experience in virtual environments. Sensors (Switzerland) 21, 4 (feb 2021), 1–24. https://doi.org/10.3390/s21041185
[48]
Jan Philipp Tauscher, Alexandra Witt, Sebastian Bosse, Fabian W. Schottky, Steve Grogorick, Susana Castillo, and Marcus Magnor. 2020. Exploring neural and peripheral physiological correlates of simulator sickness. In Computer Animation and Virtual Worlds, Vol. 31. John Wiley & Sons, Ltd, Germany, 1–11. Issue 4-5. https://doi.org/10.1002/cav.1953
[49]
Christoph Tremmel, Christian Herff, Tetsuya Sato, Krzysztof Rechowicz, Yusuke Yamani, and Dean J Krusienski. 2019. Estimating cognitive workload in an interactive virtual reality environment using EEG. Frontiers in human neuroscience 13 (2019), 401.
[50]
Meng Chang Tsai, Chia Ru Chung, Chun Chuan Chen, Shih Ching Yeh, Jyun Yu Chen, Chun Han Lin, Ying Ju Chen, Ming Che Tsai, Ya Ling Wang, Chia Ju Lin, and Hsiaokuang Wu. 2021. An Intelligent Virtual-Reality System with Multi-Model Sensing for Cue-Elicited Craving in Patients with Methamphetamine Use Disorder. IEEE Transactions on Biomedical Engineering 68 (2021), 2270–2280. Issue 7. https://doi.org/10.1109/TBME.2021.3058805
[51]
Linlin Tu, Tian Hao, Chongguang Bi, and Guoliang Xing. 2020. BreathCoach: A smart in-home breathing training system with bio-feedback via VR game. Smart Health 16 (may 2020), 100090. https://doi.org/10.1016/J.SMHL.2019.100090
[52]
Martine J. van Bennekom, Pelle P. de Koning, Martin J. Gevonden, M. Soemiati Kasanmoentalib, and Damiaan Denys. 2021. A Virtual Reality Game to Assess OCD Symptoms. Frontiers in Psychiatry 11 (jan 2021), NA–NA. https://doi.org/10.3389/fpsyt.2020.550165
[53]
Bo Wan, Qi Wang, Kejia Su, Caiyu Dong, Wenjing Song, and Min Pang. 2021. Measuring the Impacts of Virtual Reality Games on Cognitive Ability Using EEG Signals and Game Performance Data. IEEE Access 9(2021), 18326–18344. https://doi.org/10.1109/ACCESS.2021.3053621
[54]
Séamas Weech, Jae Moon, and Nikolaus F. Troje. 2018. Influence of bone-conducted vibration on simulator sickness in virtual reality. PLoS ONE 13, 3 (mar 2018), e0194137–e0194137. https://doi.org/10.1371/journal.pone.0194137
[55]
Bob G. Witmer and Michael J. Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments 7, 3(1998), 225–240. https://doi.org/10.1162/105474698565686
[56]
Hui Zhang. 2017. Head-mounted display-based intuitive virtual reality training system for the mining industry. International Journal of Mining Science and Technology 27, 4 (jul 2017), 717–722. https://doi.org/10.1016/j.ijmst.2017.05.005
[57]
Pamela Zontone, Antonio Affanni, Riccardo Bernardini, Alessandro Piras, and Roberto Rinaldo. 2019. Stress detection through Electrodermal Activity (EDA) and Electrocardiogram (ECG) analysis in car drivers. In European Signal Processing Conference, Vol. 2019-September. IEEE, A Coruna, Spain, 1–5. https://doi.org/10.23919/EUSIPCO.2019.8902631

Cited By

View all
  • (2024)Adaptive Virtual Reality Solutions: A Literature ReviewProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691574(1-10)Online publication date: 30-Sep-2024
  • (2024)Closing the Loop: The Effects of Biofeedback Awareness on Physiological Stress Response Using Electrodermal Activity in Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650830(1-7)Online publication date: 11-May-2024
  • (2024)Multisensory experiences of affective touch in virtual reality enhance engagement, body ownership, pleasantness, and arousal modulationVirtual Reality10.1007/s10055-024-01056-228:4Online publication date: 18-Oct-2024
  • Show More Cited By

Index Terms

  1. Virtual Reality Assessment and Customization Using Physiological Measures: A Literature Analysis
          Index terms have been assigned to the content through auto-classification.

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Other conferences
          SVR '21: Proceedings of the 23rd Symposium on Virtual and Augmented Reality
          October 2021
          196 pages
          ISBN:9781450395526
          DOI:10.1145/3488162
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 03 January 2022

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. customization
          2. physiological measures
          3. usability
          4. virtual reality

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Funding Sources

          • NSERC

          Conference

          SVR'21
          SVR'21: Symposium on Virtual and Augmented Reality
          October 18 - 21, 2021
          Virtual Event, Brazil

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)131
          • Downloads (Last 6 weeks)12
          Reflects downloads up to 14 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)Adaptive Virtual Reality Solutions: A Literature ReviewProceedings of the 26th Symposium on Virtual and Augmented Reality10.1145/3691573.3691574(1-10)Online publication date: 30-Sep-2024
          • (2024)Closing the Loop: The Effects of Biofeedback Awareness on Physiological Stress Response Using Electrodermal Activity in Virtual RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650830(1-7)Online publication date: 11-May-2024
          • (2024)Multisensory experiences of affective touch in virtual reality enhance engagement, body ownership, pleasantness, and arousal modulationVirtual Reality10.1007/s10055-024-01056-228:4Online publication date: 18-Oct-2024
          • (2023)Human Factors/Ergonomics (HFE) Evaluation in the Virtual Reality Environment: A Systematic ReviewInternational Journal of Human–Computer Interaction10.1080/10447318.2023.222783540:17(4533-4549)Online publication date: 29-Jun-2023
          • (2023)Immersion and Presence in Virtual Reality Applications for Physical Therapy and Upper Limb RehabilitationSocial Computing and Social Media10.1007/978-3-031-35927-9_16(217-227)Online publication date: 23-Jul-2023

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format.

          HTML Format

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media