Skip to main content
SpringerLink
Log in

Real-Time Affective Measurements in Medical Education, Using Virtual and Mixed Reality

  • Conference paper
  • First Online:
Brain Function Assessment in Learning (BFAL 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12462))

Included in the following conference series:

Abstract

Emotion greatly affects learning. Affective states, such as motivation, interest and attention, have been identified to cause changes in brain and body activity. Heart Rate (HR), Electro-Dermal Activity (EDA), and Electroencephalography (EEG) reflect physiological expressions of the human body that change according to emotional changes. In reverse, changes of bio-signal recordings can be linked to emotional changes. Virtual/Mixed Reality (V/MR) applications can be used in medical education to enhance learning. This work is a proof of application study of wearable, bio-sensor based affect detection in a learning processes, that includes the Microsoft HoloLens V/MR platform. Wearable sensors for HR, EDA and EEG signals recordings were used during two educational scenarios run by a medical doctor. The first was a conventional scenario-based Virtual Patient case for the participant’s bio-signal baselines canonization. The second was a V/MR exploratory educational neuroanatomy resource. After pre-processing and averaging, the HR and EDA recordings displayed a considerable increase during the V/MR case against the baseline. The alpha rhythm, of the EEG, had a borderline degrease and the theta over beta ration a borderline increase. These results indicate an increased attention/concentration state. They also demonstrate that the usage of bio-sensors assist in the detection the emotional state and could provide real-time, affective learning analytics using V/MR in medical education.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bloom, B.: Bloom’s taxonomy of learning domains: the three types of learning (1956). http://www.nwlink.com/~donclark/hrd/bloom.html. Accessed 13 May 2018

  2. Krathwohl, D.R., Bloom, B.S., Masla, B.B.: Taxonomy of Educational Objectives, the Classification of Educational Goals. Handbook II: Affective Domain. David McKay Co., Inc, New York (1964)

    Google Scholar 

  3. Bamidis, P.D.: Affective learning: principles, technologies, practice. BFAL 2017. LNCS (LNAI), vol. 10512, pp. 1–13. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67615-9_1

    Chapter  Google Scholar 

  4. Picard, R.W., et al.: Affective learning—a manifesto. BT Technol. J. 22(4), 253–269 (2004)

    Article  Google Scholar 

  5. Stein, N.L., Levine, L.J.: Making sense out of emotion. In: Kessen, W., Ortony, A., Kraik, F. (eds.) Memories, Thoughts, and Emotions: Essays in Honor of George Mandler, pp. 295–322. Erlbaum, Hillsdale (1991)

    Google Scholar 

  6. Mehrabian, A., Russell, J.A.: An Approach to Environmental Psychology. MIT Press, Cambridge (1974)

    Google Scholar 

  7. Russel, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980). http://psycnet.apa.org/record/1981-25062-001

  8. Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17(3), 715–734 (2005)

    Article  Google Scholar 

  9. Bakker, I., van der Voordt, T., Vink, P., de Boon, J.: Pleasure, arousal, dominance: Mehrabian and Russell revisited. Curr. Psychol. 33, 405–421 (2014)

    Article  Google Scholar 

  10. Chang, C.-Y. Zheng, J.-Y., Wang, C.-J.: Based on support vector regression for emotion recognition using physiological signals. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE. https://doi.org/10.1109/IJCNN.2010.5596878

  11. Niu, X., Chen, L., Chen, Q.: Research on genetic algorithm based on emotion recognition using physiological signals. In: 2011 International Conference on Computational Problem-Solving (ICCP), pp. 614–618. IEEE (2011). https://doi.org/10.1109/ICCPS.2011.6092256

  12. Lane, R., Reiman, E.M., Ahern, G.L., Schwartz, G.E., Davidson, R.J.: Neuroanatomical correlates of happiness, sadness and disgust. Am. J. Psychiatry 154, 926–933 (1997)

    Article  Google Scholar 

  13. Damasio, R., et al.: Subcortical and cortical brain activity during the feeling of self-generated emotions. Nat. Neurosci. 3, 1049–1056 (2000)

    Article  Google Scholar 

  14. Phelps, E.A., O’Connor, K.J., Gatenby, J.C., Gore, J.C., Grillon, C., Davis, M.: Activation of the left amygdala to a cognitive representation of fear. Nat. Neurosci. 4, 437–441 (2001)

    Article  Google Scholar 

  15. Lanteaume, L., Khalfa, S., Régis, J., Marquis, P., Chauvel, P., Bartolomei, F.: Emotion induction after direct intracerebral stimulations of human amygdala. Cereb. Cortex 17, 1307–1313 (2007)

    Article  Google Scholar 

  16. Markowitsch, J.H.: Differential contribution of right and left amygdala to affective information processing. Behav. Neurol. 11, 233–244 (1998)

    Article  Google Scholar 

  17. Paré, D., Collins, D.R., Pelletier, J.G.: Amygdala oscillations and the consolidation of emotional memories. Trends Cogn. Sci. 6, 306–314 (2002)

    Article  Google Scholar 

  18. Bamidis, P.D., Dimitrova, V., Treasure-Jones, T., Poulton, T., Roberts, T.: Augmented minds: technology’s role in supporting 21st century doctors. In: Workshop on European TEL for Workplace Learning and Professional Development (TEL@ Work). Leeds (2017)

    Google Scholar 

  19. Dafli, E., Fountoukidis, I., Hatzisevastou-Loukidou, C., Bamidis, P.D.: Curricular integration of virtual patients: a unifying perspective of medical teachers and students. BMC Med. Educ. 19(1), 416 (2019)

    Article  Google Scholar 

  20. Williams, B.: Case based learning—a review of the literature: is there scope for this educational paradigm in prehospital education? Emerg. Med. J. 22(8), 577–581 (2005). https://doi.org/10.1136/emj.2004.022707 [Medline: 16046764]

  21. Larson, J.R.: In Search of Synergy in Small Group Performance. Psychology Press, New York (2010)

    Google Scholar 

  22. Nexus-10 product page. https://www.mindmedia.com/en/products/nexus-10-mkii/

  23. Empatica E4 product page. https://www.empatica.com/en-eu/research/e4/

  24. Microsoft Hololens product page. https://www.microsoft.com/en-us/hololens

  25. Debut Video Capture Software product page. http://www.nchsoftware.com/capture/index.html

  26. Dafli, E.L., Vegoudakis, K.I., Pappas, C., Bamidis, P.D.: Re-purposing cardiology to psychiatry sessions in a 3D based virtual learning environment. Bio-Algorithms Med-Systems 6(11), 9–13 (2010)

    Google Scholar 

  27. Karrasch, M., Laine, M., Rapinoja, P., Krause, C.M.: Effects of normal aging on event-related desynchronization/synchronization during a memory task in humans. Neurosci. Lett. 366(1), 18–23 (2004)

    Article  Google Scholar 

  28. Antoniou, P.E., Sidiropoulos, E.A., Bamidis, P.D.: DISCOVER-ing beyond OpenSim; immersive learning for carers of the elderly in the VR/AR era. In: Beck, D., et al. (eds.) iLRN 2017. CCIS, vol. 725, pp. 189–200. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60633-0_16

    Chapter  Google Scholar 

  29. Antoniou, P.E., Athanasopoulou, C.A., Dafli, E., Bamidis, P.D.: Exploring design requirements for repurposing dental virtual patients from the web to second life: a focus group study. J. Med. Internet Res. 16(6), 1–19 (2014)

    Article  Google Scholar 

  30. Antoniou, P.E., Dafli, E., Arfaras, G., Bamidis, P.D.: Versatile mixed reality medical educational spaces; requirement analysis from expert users. Pers. Ubiquit. Comput. 21(6), 1015–1024 (2017). https://doi.org/10.1007/s00779-017-1074-5

    Article  Google Scholar 

  31. Charalambous, E.F., Hanna, S.E., Penn, A.L.: Visibility analysis, spatial experience and EEG recordings in virtual reality environments: the experience of ‘knowing where one is’ and isovist properties as a means to assess the related brain activity. In: Proceedings of the 11th Space Syntax Symposium 3 July 2017, vol. 11, p. 128-1. Instituto Superior Técnico, Departamento de Engenharia Civil, Arquitetura e Georrecursos, Portugal

    Google Scholar 

  32. Lin, C.-T., Chung, I.-F., Ko, L.-W., Chen, Y.-C., Liang, S.-F., Duann, J.-R.: EEG-based assessment of driver cognitive responses in a dynamic virtual-reality driving environment. IEEE Trans. Biomed. Eng. 54(7), 1349–1352 (2007)

    Article  Google Scholar 

  33. Klimesch, W.: EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Rev. 29(2–3), 169–195 (1999)

    Article  Google Scholar 

  34. Klopfer, E., Squire, K.: Environmental detectives—the development of an augmented reality platform for environmental simulations. Educ. Technol. Res. Dev. 56(2), 203–228 (2008)

    Article  Google Scholar 

  35. Kubota, Y., et al.: Frontal midline theta rhythm is correlated with cardiac autonomic activities during the performance of an attention demanding meditation procedure. Cogn. Brain Res. 11(2), 281–287 (2001)

    Article  Google Scholar 

  36. Başar-Eroglu, C., Başar, E., Demiralp, T., Schürmann, M.: P300-response: possible psychophysiological correlates in delta and theta frequency channels. A review. Int. J. Psychophysiol. 13(2), 161–179 (1992)

    Article  Google Scholar 

  37. Lagopoulos, J., et al.: Increased theta and alpha EEG activity during nondirective meditation. J. Altern. Complement. Med. 15(11), 1187–1192 (2009)

    Article  Google Scholar 

  38. Gruzelier, J.: A theory of alpha/theta neurofeedback, creative performance enhancement, long distance functional connectivity and psychological integration. Cogn. Process. 10(1 SUPPL.), 101–109 (2009)

    Article  Google Scholar 

  39. Jensen, O., Tesche, C.D.: Frontal theta activity in humans increases with memory load in a working memory task. Eur. J. Neurosci. 15(8), 1395–1399 (2002)

    Article  Google Scholar 

  40. Grunwald, M., et al.: Theta power in the EEG of humans during ongoing processing in a haptic object recognition task. Cogn. Brain Res. 11(1), 33–37 (2001)

    Article  Google Scholar 

  41. Gevins, A.S., et al.: EEG patterns during ‘cognitive’ tasks. I. Methodology and analysis of complex behaviors. Electroencephalogr. Clin. Neurophysiol. 47, 693–703 (1979)

    Google Scholar 

  42. Gevins, A.S., Zeitlin, G.M., Doyle, J.C., Schaffer, R.E., Callaway, E.: EEG patterns during ‘cognitive’ tasks. II. Analysis of controlled tasks. Electroencephalogr. Clin. Neurophysiol. 47, 704–710 (1979)

    Google Scholar 

  43. Gevins, A.S., et al.: Electroencephalogram correlates of higher cortical functions. Science 203(4381), 665–668 (1979)

    Article  Google Scholar 

  44. Adrian, E.D., Matthews, B.H.: The interpretation of potential waves in the cortex. J. Physiol. 81(4), 440–471 (1934)

    Article  Google Scholar 

  45. Niedermeyer, E., Lopes da Silva, F.: Electroencephalography: Basic Principles, Clinical Applications, and Related Fields, 5th edn. Williams & Wilkins, Baltimore (2004)

    Google Scholar 

  46. Fowles, D.C.: The three arousal model: implications of Gray’s two-factor learning theory for heart rate, electrodermal activity, and psychopathy. Psychophysiology 17(2), 87–104 (1980)

    Article  Google Scholar 

  47. Drachen, A., Nacke, L.E., Yannakakis, G., Pedersen, A.L.: Correlation between heart rate, electrodermal activity and player experience in first-person shooter games. In: Proceedings of the 5th ACM SIGGRAPH Symposium on Video Games 28 July 2010, pp. 49–54. ACM (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lab of Medical Physics, School of Medicine, Faculty of Health Sciences, Aristotle University of Thessaloniki, Thessaloniki, Greece

    Panagiotis Antoniou, George Arfaras, Niki Pandria, George Ntakakis, Emmanuil Bambatsikos & Alkinoos Athanasiou

Authors
  1. Panagiotis Antoniou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George Arfaras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Niki Pandria
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. George Ntakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emmanuil Bambatsikos
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alkinoos Athanasiou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panagiotis Antoniou .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of Montreal, Montreal, QC, Canada

    Claude Frasson

  2. Lab of Medical Physics, Aristotle University of Thessaloniki, Thessaloniki, Greece

    Panagiotis Bamidis

  3. Department of Informatics, Ionian University, Corfu, Greece

    Panagiotis Vlamos

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Antoniou, P., Arfaras, G., Pandria, N., Ntakakis, G., Bambatsikos, E., Athanasiou, A. (2020). Real-Time Affective Measurements in Medical Education, Using Virtual and Mixed Reality. In: Frasson, C., Bamidis, P., Vlamos, P. (eds) Brain Function Assessment in Learning. BFAL 2020. Lecture Notes in Computer Science(), vol 12462. Springer, Cham. https://doi.org/10.1007/978-3-030-60735-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60735-7_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60734-0

  • Online ISBN: 978-3-030-60735-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation