Abstract
Virtual reality (VR) is getting traction in many contexts, allowing users to have a real-life experience in a virtual world. However, its application in the field of Neuroscience, and above all probing newer activity with the analysis of electroencephalographic (EEG) event-related potentials (ERP) is underexplored. This article reviews the state-of-the-art applications of virtual reality in ERP research, analysing current ways to integrate Head-Mounted Displays (HMD) with EEG head-sets for deploying ecologically valid experiments. It also identifies which ERP components are appropriate in VR settings, along with their paradigms, the technical configurations of the experiments conducted, and the reliability of the findings. Finally, the article synthesises this survey, providing recommendations to practitioners and scholars.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahn, M.H., Park, J.H., Jeon, H., Lee, H.J., Kim, H.J., Hong, S.K.: Temporal dynamics of visually induced motion perception and neural evidence of alterations in the motion perception process in an immersive virtual reality environment. Front. Neurosci. 14 (2020)
Aksoy, M., Ufodiama, C.E., Bateson, A.D., Martin, S., Asghar, A.U.R.: A comparative experimental study of visual brain event-related potentials to a working memory task: virtual reality head-mounted display versus a desktop computer screen. Exp. Brain Res. 239(10), 3007–3022, 104107 (2021). https://doi.org/10.1007/s00221-021-06158-w
Arake, M., et al.: Measuring task-related brain activity with event-related potentials in dynamic task scenario with immersive virtual reality environment. Front. Behav. Neurosci. 16, 11 (2022)
Baumgartner, T., Valko, L., Esslen, M., Jäncke, L.: Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study. CyberPsychol. Behav. 9(1), 30–45 (2006)
Burns, C.G., Fairclough, S.H.: Use of auditory event-related potentials to measure immersion during a computer game. Int. J. Hum Comput Stud. 73, 107–114 (2015)
Cattan, G., Andreev, A., Visinoni, E.: Recommendations for integrating a p300-based brain-computer interface in virtual reality environments for gaming: an update. Computers 9(4), 92 (2020)
Dey, A., Chatburn, A., Billinghurst, M.: Exploration of an EEG-based cognitively adaptive training system in virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 220–226. IEEE (2019)
Du, J., Ke, Y., Kong, L., Wang, T., He, F., Ming, D.: 3D stimulus presentation of ERP-speller in virtual reality. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 167–170. IEEE (2019)
Erdogdu, E., Kurt, E., Duru, A.D., Uslu, A., Başar-Eroğlu, C., Demiralp, T.: Measurement of cognitive dynamics during video watching through event-related potentials (ERPS) and oscillations (EROS). Cogn. Neurodyn. 13(6), 503–512 (2019)
Garduno Luna, C.D.: Feasibility of virtual and augmented reality devices as psychology research tools: a pilot study. Ph.D. thesis, UC Santa Barbara (2020)
Gehrke, L., et al.: Detecting visuo-haptic mismatches in virtual reality using the prediction error negativity of event-related brain potentials. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2019)
Grassini, S., Laumann, K., Thorp, S., Topranin, V.d.M.: Using electrophysiological measures to evaluate the sense of presence in immersive virtual environments: an event-related potential study. Brain Behav. 11(8), e2269 (2021)
Hajcak, G., Klawohn, J., Meyer, A.: The utility of event-related potentials in clinical psychology. Annu. Rev. Clin. Psychol. 15, 71–95 (2019)
Harjunen, V.J., Ahmed, I., Jacucci, G., Ravaja, N., Spapé, M.M.: Manipulating bodily presence affects cross-modal spatial attention: a virtual-reality-based ERP study. Front. Hum. Neurosci. 11, 79 (2017)
Herweg, A., Gutzeit, J., Kleih, S., Kübler, A.: Wheelchair control by elderly participants in a virtual environment with a brain-computer interface (BCI) and tactile stimulation. Biol. Psychol. 121, 117–124 (2016)
Hou, G., Dong, H., Yang, Y.: Developing a virtual reality game user experience test method based on EEG signals. In: 2017 5th International Conference on Enterprise Systems (ES), pp. 227–231. IEEE (2017)
Hubbard, R., Sipolins, A., Zhou, L.: Enhancing learning through virtual reality and neurofeedback: a first step. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 398–403 (2017)
Hyun, K.Y., Lee, G.H.: Analysis of change of event related potential in escape test using virtual reality technology. Biomed. Sci. Lett. 25(2), 139–148 (2019)
Käthner, I., Kübler, A., Halder, S.: Rapid p300 brain-computer interface communication with a head-mounted display. Front. Neurosci. 9, 207 (2015)
Kim, S., Lee, S., Kang, H., Kim, S., Ahn, M.: P300 brain-computer interface-based drone control in virtual and augmented reality. Sensors 21(17), 5765 (2021)
Kirasirova, L., Zakharov, A., Morozova, M., Kaplan, A.Y., Pyatin, V.: ERP correlates of emotional face processing in virtual reality. Opera Med. Physiol. 8(3), 12–19 (2021)
Kober, S.E., Neuper, C.: Using auditory event-related EEG potentials to assess presence in virtual reality. Int. J. Hum Comput Stud. 70(9), 577–587 (2012)
Li, G., Zhou, S., Kong, Z., Guo, M.: Closed-loop attention restoration theory for virtual reality-based attentional engagement enhancement. Sensors 20(8), 2208 (2020)
Liang, S., Choi, K.S., Qin, J., Pang, W.M., Wang, Q., Heng, P.A.: Improving the discrimination of hand motor imagery via virtual reality based visual guidance. Comput. Methods Programs Biomed. 132, 63–74 (2016)
Lin, C.T., Chung, I.F., Ko, L.W., Chen, Y.C., Liang, S.F., Duann, J.R.: EEG-based assessment of driver cognitive responses in a dynamic virtual-reality driving environment. IEEE Trans. Biomed. Eng. 54(7), 1349–1352 (2007)
Ogawa, R., Kageyama, K., Nakatani, Y., Ono, Y., Murakami, S.: Event-related potentials-based evaluation of attention allocation while watching virtual reality. Adv. Biomed. Eng. 11, 1–9 (2022)
Pavone, E.F., Tieri, G., Rizza, G., Tidoni, E., Grisoni, L., Aglioti, S.M.: Embodying others in immersive virtual reality: electro-cortical signatures of monitoring the errors in the actions of an avatar seen from a first-person perspective. J. Neurosci. 36(2), 268–279 (2016)
Peeters, D.: Bilingual switching between languages and listeners: insights from immersive virtual reality. Cognition 195, 104107 (2020)
Peterson, S.M., Furuichi, E., Ferris, D.P.: Effects of virtual reality high heights exposure during beam-walking on physiological stress and cognitive loading. PLoS ONE 13(7) (2018)
Petras, K., Ten Oever, S., Jansma, B.M.: The effect of distance on moral engagement: Event related potentials and alpha power are sensitive to perspective in a virtual shooting task. Front. Psychol. 6, 2008 (2016)
Pezzetta, R., Nicolardi, V., Tidoni, E., Aglioti, S.M.: Error, rather than its probability, elicits specific electrocortical signatures: a combined EEG-immersive virtual reality study of action observation. J. Neurophysiol. 120(3), 1107–1118 (2018)
Picton, T.W., et al.: Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37(2), 127–152 (2000)
Schubring, D., Kraus, M., Stolz, C., Weiler, N., Keim, D.A., Schupp, H.: Virtual reality potentiates emotion and task effects of alpha/beta brain oscillations. Brain Sci. 10(8), 537 (2020)
Si-Mohammed, H., et al.: Detecting system errors in virtual reality using EEG through error-related potentials. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 653–661. IEEE (2020)
Simões, M.., Amaral, C.., Carvalho, Paulo, Castelo-Branco, Miguel: Specific EEG/ERP responses to dynamic facial expressions in virtual reality environments. In: Zhang, Yuan-Ting. (ed.) The International Conference on Health Informatics. IP, vol. 42, pp. 331–334. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-03005-0_84
Singh, A.K., Chen, H.T., Cheng, Y.F., King, J.T., Ko, L.W., Gramann, K., Lin, C.T.: Visual appearance modulates prediction error in virtual reality. IEEE Access 6, 24617–24624 (2018)
Sun, R., Wu, Y.J., Cai, Q.: The effect of a virtual reality learning environment on learners’ spatial ability. Virtual Reality 23(4), 385–398 (2019)
Tauscher, J.P., Schottky, F.W., Grogorick, S., Bittner, P.M., Mustafa, M., Magnor, M.: Immersive EEG: evaluating electroencephalography in virtual reality. In: 2019 Conference on Virtual Reality and 3D User Interfaces, pp. 1794–1800. IEEE (2019)
de Tommaso, M., et al.: Pearls and pitfalls in brain functional analysis by event-related potentials: a narrative review by the Italian psychophysiology and cognitive neuroscience society on methodological limits and clinical reliability-Part I. Neurol. Sci. 41, 3503–3515 (2020)
Török, Á., et al.: Comparison between wireless and wired EEG recordings in a virtual reality lab: Case report. In: 2014 5th Conference on Cognitive Infocommunications (CogInfoCom), pp. 599–603. IEEE (2014)
Tosoni, A., Altomare, E.C., Brunetti, M., Croce, P., Zappasodi, F., Committeri, G.: Sensory-motor modulations of EEG event-related potentials reflect walking-related macro-affordances. Brain Sci. 11(11), 1506 (2021)
Vass, L.K., et al.: Oscillations go the distance: low-frequency human hippocampal oscillations code spatial distance in the absence of sensory cues during teleportation. Neuron 89(6), 1180–1186 (2016)
Wu, J., Zhou, Q., Li, J., Kong, X., Xiao, Y.: Inhibition-related n2 and p3: Indicators of visually induced motion sickness (vims). Int. J. Ind. Ergon. 78, 102981 (2020)
Yokota, Y., Naruse, Y.: Temporal fluctuation of mood in gaming task modulates feedback negativity: Eeg study with virtual reality. Front. Hum. Neurosci. 15, 246 (2021)
Acknowledgement
This work was conducted with the financial support of the Science Foundation Ireland Centre for Research Training in Digitally-Enhanced Reality (D-real) under Grant No. 18/CRT/6224.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Marochko, V., Reilly, R., McDonnell, R., Longo, L. (2022). A Survey on the Application of Virtual Reality in Event-Related Potential Research. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds) Machine Learning and Knowledge Extraction. CD-MAKE 2022. Lecture Notes in Computer Science, vol 13480. Springer, Cham. https://doi.org/10.1007/978-3-031-14463-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-14463-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14462-2
Online ISBN: 978-3-031-14463-9
eBook Packages: Computer ScienceComputer Science (R0)