Skip to main content
Springer Nature Link
Log in

Pleasure–arousal–outlier model for quantitative evaluation of game experiences

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This study proposes a pleasure–arousal–outlier (PAO) model to quantify the experiences derived from games. The proposed technique identifies pleasure, arousal, and outlier levels based on the facial expression of a user, keyboard input information, and mouse movement information received from a multimodal interface and then projects the received information in three-dimensional space to quantify the game experience state of the user. Facial expression recognition and distribution, eye blink, and eye glance concentration graphs were introduced to determine the immersion levels of games. A convolutional neural network-based facial expression recognition algorithm and dynamic time warp-based outlier behavior detection algorithm were adopted to obtain numerical values required for the PAO model evaluation. We applied the proposed PAO model for first-person shooter games and consequently acquired evaluation result values that were clearly distinguishable for different players. Such information allows player experiences to be quantitatively evaluated when designing game levels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hudlicka E (2008) Affective computing for game design. In: Proceedings of the 4th Intl. North American Conference on Intelligent Games and Simulation, pp 5–12

  2. Lee E, Jang Y, Yoon D, Jeon J, Yang SI, Lee SK, Kim KJ (2008) Understanding customer malling behavior in an urban shopping mall using smartphones. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication pp 901–910

  3. Ahn J, Hwang J, Kim D, Choi H, Kang S (2008) A survey on churn analysis in various business domains. IEEE Access 8:220816–220839

    Article  Google Scholar 

  4. Chung Y, Park CY, Kim NR, Cho H, Yoon T, Lee H, Lee JH (2013) Game bot detection approach based on behavior analysis and consideration of various play styles. ETRI J 35(6):1058–1067

    Article  Google Scholar 

  5. Bänziger T, Scherer KR (2007) Using actor portrayals to systematically study multimodal emotion expression: the GEMEP corpus. Lect Notes Comput Sci 4738:476–487

    Article  Google Scholar 

  6. Petrantonakis Panagiotis C, Leontios J (2014) EEG-based emotion recognition using advanced signal processing techniques. A Pattern Analysis Approach, Emotion Recognition, pp 269–293

    Google Scholar 

  7. Wang Y, Agrafioti F, Hatzinakos D, Plataniotis KN (2008) Analysis of human electrocardiogram for biometric recognition. EURASIP J Adv Signal Process 2008:11

    MATH  Google Scholar 

  8. Wagner J, Kim JH, Andre E (2005) From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification, multimedia and expo. IEEE International Conference on Multimedia and Expo, pp 940–943

  9. Leon E, Clarke G, Callaghan V, Sepulveda F (2007) A user-independent real-time emotion recognition system for software agents in domestic environments. Eng Appl Artif Intell 20(3):337–345

    Article  Google Scholar 

  10. McDuff D, Gontarek S, Picard RW (2014) Remote measurement of cognitive stress via heart rate variability. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

  11. Hernandez J, Picard RW (2014) SenseGlass: using google glass to sense daily emotions, acm user interface software and technology symposium, Honolulu, Hawaii, October 2014

  12. Ekman P, Rosenberg EL (1997) What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS). Oxford University Press, England

    Google Scholar 

  13. Kapoor a, Burleson W, Picard RW (2005) Multimodal Affect recognition in learning environments. In: ACM International Conference on Multimedia, 2005; pp 672–682

  14. El Kaliouby R, Teeters A, Picard RW (2006) An exploratory social-emotional prosthetic for autism spectrum disorders, wearable and implantable body sensor networks. International workshop on wearable and implantable body sensor networks (BSN’06), 2006; pp 2–4

  15. Littlewort GC, Bartlett MS, Lee K (2007) Faces of pain: automated measurement of spontaneous facial expressions of genuine and posed pain. In: Proceedings of the 9th International Conference on Multimodal Interfaces; pp 15–21

  16. Yeasin M, Sharma Bullot BR (2006) Recognition of facial expressions and measurement of levels of interest from video. IEEE Transact Multimed 8(3):500–507

    Article  Google Scholar 

  17. Nosu K, Kurokawa T, Horita H, Ohhazama Y (2007) Real time emotion-diagnosis of video game players from their facial expressions and its applications to voice feed-backing to game players. In: Proceeding of International Conference on Machine Learning and Cybernetics, 2007; vol 4

  18. Vachirapanang K, Tuisima S, Sinthupinyo S, Sirivunnabood P (2012) The classification of the real-time interaction-based behavior of online game addiction in children and early adolescents in Thailand. Int J Adv Res Artif Intell 1(7):7–13

    Google Scholar 

  19. Yun C, Shastri D, Pavlidis I, Deng Z (2009) O ’ game, can you feel my frustration? Improving user’s gaming experience via stresscam. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2009: 2195–2204

  20. Tan CT, Rosser D, Bakkes S, Pisan Y (2012) A feasibility study in using facial expressions analysis to evaluate player experiences. Aust Conf Interact Entertain 2012:5

    Google Scholar 

  21. Hazelett RL (2003) Measurement of user frustration: a biologic approach. In: Proceedings of the Conference on Human Factors in Computing Systems 2003:734–735

  22. Tan CSS, Schöning J, Barnes JS, Luyten K, Coninx K (2013) Bro-cam: improving game experience with empathic feedback using posture tracking. Persuasive technology, lecture notes in computer science. 7822(2013):222–233

  23. Zoeller G (2010) Game development telemetry. In: Proceedings of the Game Developers Conference

  24. Martinez HP, Bengio Y, Yannakakis GN (2013) Learning deep physiological models of affect. IEEE Comput Intell Mag 8(2):20–33

    Article  Google Scholar 

  25. Wang X, Goh DHL (2020) Components of game experience: an automatic text analysis of online reviews. Entertain Comput 33:100338

    Article  Google Scholar 

  26. Maman L, Ceccaldi E, Lehmann-Willenbrock N, Likforman-Sulem L, Chetouani M, Volpe G, Varni G (2020) GAME-ON: a multimodal dataset for cohesion and group analysis. IEEE Access 8:124185–124203

    Article  Google Scholar 

  27. Song M, Yang Z, Baird A, Parada-Cabaleiro E, Zhang Z, Zhao Z, Schuller B (2019) Audiovisual analysis for recognizing frustration during game-play: introducing the multimodal game frustration database. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), 2019; pp 517–523. IEEE

  28. Ringer C, Walker JA, Nicolaou MA (2019) Multimodal joint emotion and game context recognition in league of legends livestreams. IEEE Conference on Games (CoG), 2019; pp 1–8, IEEE

  29. Sekhavat YA, Roohi S, Mohammadi HS, Yannakakis GN (2020) Play with one’s feelings: a study on emotion awareness for player experience. IEEE Transact Games

  30. Kar R, Ghosh L, Konar A, Chakraborty A, Nagar AK (2021) EEG-induced autonomous game-teaching to a robot arm by human trainers using reinforcement learning. IEEE Transact Games

  31. Bales RF (2017) Social interaction systems: theory and measurement. Routledge

    Book  Google Scholar 

  32. Kang S, Kim D, Kim Y (2019) A visual-physiology multimodal system for detecting outlier behavior of participants in a reality TV show. Int J Distrib Sens Netw 15(7):1550147719864886

    Article  Google Scholar 

  33. Gu H, Ji Q (2004) An automated face reader for fatigue detection. 6th IEEE International Conference on Automatic Face and Gesture Recognition, pp 111–116

  34. Dlib, http://dlib.net/

  35. Weste N, Burr DJ, Ackland BD (1983) Dynamic time warp pattern matching using an integrated multiprocessing array. IEEE Trans Comput 32(8):731–744

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (No. NRF-2019R1A2C1002525).

Author information

Authors and Affiliations

  1. Hongik University, Sejong, 30016, Korea

    Shinjin Kang

  2. Jeju National University, Jeju, 63243, Korea

    Soo Kyun Kim

Authors
  1. Shinjin Kang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Soo Kyun Kim
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Soo Kyun Kim.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest to declare regarding this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, S., Kim, S. Pleasure–arousal–outlier model for quantitative evaluation of game experiences. J Supercomput 78, 19459–19477 (2022). https://doi.org/10.1007/s11227-022-04636-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04636-8

Keywords