Abstract
In this paper, we present a solution for computer assisted emotional analysis of game session. The proposed approach combines eye movements and facial expressions to annotate the perceived game objects with the expressed dominate emotions. Moreover, our system EMOGRAPH (Emotional Graph) gives easy access to information about user experience and predicts player’s emotions. The prediction mainly uses both subjective measures through questionnaire and objective measures through brain wave activity (electroencephalography - EEG) combined with eye tracking data. EMOGRAPH’s method was experimented on 21 participants playing horror game “Outlast”. Our results show the effectiveness of our method in the identification of the emotions and their triggers. We also present our emotion prediction approach using game scene’s design goal (defined by OCC variables from the model of emotions’ cognitive evaluation of Ortony, Clore and Collins [1]) to annotate the player’s situation in a scene and machine learning algorithms. The prediction results are promising and would widen possibilities in game design.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1990)
Harley, J.M.: Measuring emotions: a survey of cutting edge methodologies used in computer-based learning environment research. Elsevier (2016)
Hemenover, S.H., Bowman, N.D.: Video games, emotion, and emotion regulation: expanding the scope. Ann. Int. Commun. Assoc. 42, 1–19 (2018)
Villani, D., Carissoli, C., Triberti, S., Marchetti, A., Gilli, G., Riva, G.: Videogames for emotion regulation: a systematic review. Games Health J. 7(2), 85–99 (2018)
Abdessalem, H.B., Frasson, C.: Real-time brain assessment for adaptive virtual reality game: a neurofeedback approach. Brain Function Assessment in Learning. LNCS (LNAI), vol. 10512, pp. 133–143. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67615-9_12
Gratch, J., Marsella, S., Petta, P.: Modeling the cognitive antecedents and consequences of emotion. Cognit. Syst. Res. 10(1), 1–5 (2009)
Scherer, K.R.: Vocal affect expression: a review and a model for future research. Psychol. Bull. 99(2), 143 (1986)
Lazarus, R.S.: Emotion and Adaptation. Cité en1991, 9 (1991)
Elliot, A.J., Covington, M.V.: Approach and avoidance motivation. Educ. Psychol. Rev. 13(2), 73–92 (2001)
Davidson, R.J.: What does the prefrontal cortex “do” in affect: perspectives on frontal EEG asymmetry research. Biol. Psychol. 67(1), 219–234 (2004)
Amodio, D.M., Master, S.L., Yee, C.M., Taylor, S.E.: Neurocognitive components of the behavioral inhibition and activation systems: implications for theories of self-regulation. Psychophysiology 45(1), 11–19 (2008)
Derbali, L., Ghali, R., Frasson, C.: Assessing motivational strategies in serious games using hidden markov models (2013)
Derbali, L., Frasson, C.: Prediction of players motivational states using electrophysiological measures during serious game play. IEEE (2010)
Kors, M.J., Ferri, G., van der Spek, E.D., Ketel, C., Schouten, B.A.: A breathtaking journey. On the design of an empathy-arousing mixed-reality game. ACM (2016)
Quick, J.M., Atkinson, R.K., Lin, L.: The gameplay enjoyment model. Int. J. Gaming Comput.-Mediat. Simul. (IJGCMS) 4(4), 64–80 (2012)
Sweetser, P., Johnson, D.M., Wyeth, P.: Revisiting the GameFlow model with detailed heuristics. J.: Creat. Technol. 3 (2012)
Nacke, L.E., Kalyn, M., Lough, C., Mandryk, R.L.: Biofeedback game design: using direct and indirect physiological control to enhance game interaction. ACM (2011)
Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)
Benlamine, M., Chaouachi, M., Frasson, C., Dufresne, A.: Physiology-based recognition of facial micro-expressions using EEG and identification of the relevant sensors by emotion. In: Proceedings of the 3rd International Conference on Physiological Computing Systems, PhyCS2016, vol. 1 (2016)
Stemmler, G.: Physiological processes during emotion. In: The Regulation of Emotion, pp. 33–70 (2004)
Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., Bartlett, M.: The computer expression recognition toolbox (CERT). IEEE (2011)
Zirbel, E.L.: Learning, concept formation and conceptual change. Tufts University (2014)
LeDoux, J.E.: Brain mechanisms of emotion and emotional learning. Curr. Opin. Neurobiol. 2(2), 191–197 (1992)
LeDoux, J.E.: Semantics, surplus meaning, and the science of fear. Trends Cognit. Sci. 21(5), 303–306 (2017)
Lynch, T., Martins, N.: Nothing to fear? An analysis of college students’ fear experiences with video games. J. Broadcast. Electron. Media 59(2), 298–317 (2015)
Lin, J.-H. T., Wu, D.-Y., Tao, C.-C.: So scary, yet so fun: the role of self-efficacy in enjoyment of a virtual reality horror game. New Media Soc. 1461444817744850 (2017)
Goldberg, L.R.: The development of markers for the big-five factor structure. Psychol. Assess. 4(1), 26 (1992)
Coan, J.A., Allen, J.J.: Frontal EEG asymmetry and the behavioral activation and inhibition systems. Psychophysiology 40(1), 106–114 (2003)
Libet, B.: Reflections on the interaction of the mind and brain. Prog. Neurobiol. 78(3), 322–326 (2006)
Acknowledgments
We thank the Natural Sciences and Engineering Research Council of Canada NSERC and BMU Games for funding this research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Doumbouya, R., Benlamine, M.S., Dufresne, A., Frasson, C. (2018). Game Scenes Evaluation and Player’s Dominant Emotion Prediction. In: Nkambou, R., Azevedo, R., Vassileva, J. (eds) Intelligent Tutoring Systems. ITS 2018. Lecture Notes in Computer Science(), vol 10858. Springer, Cham. https://doi.org/10.1007/978-3-319-91464-0_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-91464-0_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91463-3
Online ISBN: 978-3-319-91464-0
eBook Packages: Computer ScienceComputer Science (R0)