Skip to main content

Human-AI Coordination to Induce Flow in Adaptive Learning Systems

  • Conference paper
  • First Online:
AI Approaches for Designing and Evaluating Interactive Intelligent Systems (ROCHI 2022)

Abstract

The coordination between humans and artificial intelligence (AI) systems has the potential to achieve outcomes that neither humans nor AI could achieve alone. AI can process large amounts of data rapidly while humans are able to make use of the AI capabilities to achieve desirable outcomes. In this chapter, we focus on the use of AI in improving user experience in adaptive learning systems. Particularly, we are concerned with whether and how AI can assist in inducing a state of flow in human users. First, we review the literature on flow and adaptive instruction. Then, we describe an experimental study aiming to test an AI agent designed to coordinate with the human user and induce a state of flow. For the experiment, we developed an interactive version of the game Tetris based on the Meta-T software. In this version, we created a balancing feedback loop intended to keep the human player in a continuous state of flow. The human plays the standard Tetris game while an AI algorithm attempts to determine the player’s skill and dynamically alters the game difficulty to match it. The experimental study pitching this adaptive condition against easy and hard conditions shows that the adaptive condition has a positive effect on a composite criterion made of 60% performance and 40% flow. Arguably, this is a realistic criterion for many human performance domains. The adaptive condition, powered by the AI algorithm, does well on this composite criterion because it avoids the pitfalls of the easy and hard conditions: the easy condition hurts performance while the hard condition hurts flow.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.sona-systems.com/.

References

  1. V. Aleven, E.A. McLaughlin, R.A. Glenn, K.R Koedinger, Instruction based on adaptive learning technologies, in Handbook of Research on Learning and Instruction, 2nd edn., ed. by R.E. Mayer, P. Alexander (Routledge, 2017), pp. 522–560

    Google Scholar 

  2. A. Alexiou, M.C. Schippers, I. Oshri, S. Angelopoulos, Narrative and aesthetics as antecedents of perceived learning in serious games. Inf. Technol. People 35(8), 142–161 (2020). https://doi.org/10.1108/ITP-08-2019-0435

    Article  Google Scholar 

  3. S. Algorta, Ö. Şimşek, The game of Tetris in machine learning (2019). https://doi.org/10.48550/arXiv.1905.01652

  4. S.M. Asish, A.K. Kulshreshth, C.W. Borst, Detecting distracted students in educational VR environments using machine learning on eye gaze data. Comput. Graph. 109, 75–87 (2022). https://doi.org/10.1016/j.cag.2022.10.007

    Article  Google Scholar 

  5. M. Barthelmäs, J. Keller, Antecedents, boundary conditions and consequences of flow, in Advances in Flow Research, ed. by C. Peifer, S. Engeser (Springer, Cham, 2021). https://doi.org/10.1007/978-3-030-53468-4_3

  6. A. Belle, R.H. Hargraves, K. Najarian, An automated optimal engagement and attention detection system using electrocardiogram. Comput. Math. Methods Med. 2012, 528781 (2012). https://doi.org/10.1155/2012/528781

    Article  MathSciNet  Google Scholar 

  7. J.H. Brockmyer, C.M. Fox, K.A. Curtiss, E. McBroom, K.M. Burkhart, J.N. Pidruzny, The development of the Game Engagement Questionnaire: a measure of engagement in video game-playing. J. Exp. Soc. Psychol. 45(4), 624–634 (2009). https://doi.org/10.1016/j.jesp.2009.02.016

    Article  Google Scholar 

  8. J.A. Cannon-Bowers, E. Salas, S. Converse, Shared mental models in expert team decision making, in Individual and Group Decision Making: Current Issues. ed. by N.J. Castellan (Lawrence Erlbaum Associates Inc., 1993), pp.221–246

    Google Scholar 

  9. G. Chanel, C. Rebetez, M. Bétrancourt, T. Pun, Boredom, engagement and anxiety as indicators for adaptation to difficulty in games, in Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era. (ACM, Oct 2008), pp. 13–17. https://doi.org/10.1145/1457199.1457203

  10. C. C. Chang, C.A. Warden, C. Liang, G.Y. Lin, Effects of digital game-based learning on achievement, flow and overall cognitive load. Aust. J. Educ. Technol. 34(4) (2018). https://doi.org/10.14742/ajet.2961

  11. X. Chen, J. Cheng, R. Song, Y. Liu, R. Ward, Z.J. Wang, Video-based heart rate measurement: Recent advances and future prospects. IEEE Trans. Instrum. Meas. 68(10), 3600–3615 (2018). https://doi.org/10.1109/TIM.2018.2879706

    Article  Google Scholar 

  12. C.A. Cruz, J.A.R. Uresti, Play-centered game AI from a flow perspective: towards a better understanding of past trends and future directions. J. Entertain. Comput. 20, 11–24 (2017). https://doi.org/10.1016/j.entcom.2017.02.003

    Article  Google Scholar 

  13. M. Csikszentmihalyi, Flow and the Foundations of Positive Psychology: The Collected Works of Mihaly Csikszentmihalyi. (Springer, 2014)

    Google Scholar 

  14. M. Csikszentmihalyi, R. Larson, Validity and reliability of the Experience-Sampling Method. J. Nerv. Ment. Dis. 175(9), 526–536 (1987). https://doi.org/10.1097/00005053-198709000-00004

    Article  Google Scholar 

  15. M. Czikszentmihalyi, Flow: The Psychology of Optimal Experience (Harper & Row, New York, 1990)

    Google Scholar 

  16. S. D’Mello, A. Graesser, Affect detection from human-computer dialogue with an intelligent tutoring system, in Intelligent virtual agents. IVA 2006, ed. by J. Gratch, M. Young, R. Aylett, D. Ballin, P. Olivier. Lecture Notes in Computer Science, vol. 4133 (Springer, Berlin, Heidelberg, 2006). https://doi.org/10.1007/11821830_5

  17. C. De Looze, S. Scherer, B. Vaughan, N. Campbell, Investigating automatic measurements of prosodic accommodation and its dynamics in social interaction. Speech Commun. 58, 11–34 (2014). https://doi.org/10.1016/j.specom.2013.10.002

    Article  Google Scholar 

  18. A. Denisova, A.I. Nordin, P. Cairns, The convergence of player experience questionnaires, in Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play (Oct 2016), pp. 33–37. https://doi.org/10.1145/2967934.2968095 

  19. P. Ekman, Expression and the nature of emotion, in Approaches to Emotion. ed. by K. Scherer, P. Ekman (Erlbaum, Hillsdale, NJ, 1984), pp.319–344

    Google Scholar 

  20. S. Engeser, F. Rheinberg, Flow, performance and moderators of challenge-skill balance. Motiv. Emot. 32(3), 158–172 (2008). https://doi.org/10.1007/s11031-008-9102-4

    Article  Google Scholar 

  21. C.P. Fahey,.Tetris AI, Computer plays Tetris (2003). https://tinyurl.com/4t35vmxk

  22. L. Farnworth, The experience sampling method: its potential use in occupational therapy research. Occup. Ther. Int. 3(1), 1–17 (1996). https://doi.org/10.1002/oti.23

    Article  Google Scholar 

  23. C.J. Fong, D.J. Zaleski, J.K. Leach, The challenge–skill balance and antecedents of flow: a meta-analytic investigation. J. Posit. Psychol. 10(5), 425–446 (2015). https://doi.org/10.1080/17439760.2014.967799

    Article  Google Scholar 

  24. J. Graft, W. Romine, B. Watts, N. Schroeder, T. Jawad, T. Banerjee, A preliminary study of the efficacy of using a wrist-worn multiparameter sensor for the prediction of cognitive flow states in university-level students. Sensors 23, 3957 (2023). https://doi.org/10.3390/s23083957

  25. D. Gunning, E. Vorm, J.Y. Wang, M. Turek, DARPA’s explainable AI (XAI) program: a retrospective. Appl. AI Lett. 2, e61 (2021). https://doi.org/10.1002/ail2.61

    Article  Google Scholar 

  26. R.J. Haier, S. Karama, L. Leyba, R.E. Jung, MRI assessment of cortical thickness and functional activity changes in adolescent girls following three months of practice on a visual-spatial task. BMC. Res. Notes 2, 174 (2009). https://doi.org/10.1186/1756-0500-2-174

    Article  Google Scholar 

  27. J.A. Hamilton, R.J. Haier, M.S. Buchsbaum, Intrinsic enjoyment and boredom coping scales: Validation with personality, evoked potential and attention measures. Personality Individ. Differ. 5(2), 183–193 (1984). https://doi.org/10.1016/0191-8869(84)90050-3

    Article  Google Scholar 

  28. D.J. Harris, S.J. Vine, M.R. Wilson, Is flow really effortless? The complex role of effortful attention. Sport Exerc. Perform. Psychol. 6(1), 103–114 (2017). https://doi.org/10.1037/spy0000083

    Article  Google Scholar 

  29. M. Haug, P. Camps, T. Umland, J.N. Voigt-Antons, Assessing differences in flow state induced by an adaptive music learning software, in 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) (IEEE, May 2020), pp. 1–4. https://doi.org/10.1109/QoMEX48832.2020.9123132 

  30. J. Heutte, F. Fenouillet, J. Kaplan, C. Martin-Krumm, R. Bachelet, The EduFlow Model: a contribution toward the study of optimal learning environments, in Flow experience: Empirical research and applications, ed. by L. Harmat, F. Ø. Andersen, F. Ullén, J. Wright, G. Sadlo (Springer International Publishing/Springer Nature, 2016), pp. 127–143. https://doi.org/10.1007/978-3-319-28634-1_9

  31. A. Holzinger, Interactive machine learning for health informatics: When do we need the human-in-the-loop? Brain Informatics 3(2), 119–131 (2016). https://doi.org/10.1007/s40708-016-0042-6

    Article  Google Scholar 

  32. X.T. Huang, Z.D. Wei, X.Y. Leung, What you feel may not be what you experience: a psychophysiological study on flow in VR travel experiences. Asia Pac. J. Tour. Res. 25(7), 736–747 (2020). https://doi.org/10.1080/10941665.2019.1711141

    Article  Google Scholar 

  33. M.B. Ibáñez, Á. Di Serio, D. Villarán, C.D. Kloos, Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Comput. Educ. 71, 1–13 (2014). https://doi.org/10.1016/j.compedu.2013.09.004

    Article  Google Scholar 

  34. S.A. Jackson, R.C. Eklund, Assessing flow in physical activity: the flow state scale–2 and dispositional flow scale–2. J. Sport Exerc. Psychol. 24(2), 133–150 (2002). https://doi.org/10.1123/jsep.24.2.133

    Article  Google Scholar 

  35. S.A. Jackson, A.J. Martin, R.C. Eklund, Long and short measures of flow: The construct validity of the FSS-2, DFS-2, and new brief counterparts. J. Sport Exerc. Psychol. 30(5), 561–587 (2008). https://doi.org/10.1123/jsep.30.5.561

    Article  Google Scholar 

  36. I. Juvina, K. O'Neill, Adaptive interface promotes a composite of performance and flow in Tetris, in The 19th International Conference on Human-Computer Interaction, RoCHI 2022, Craiova, Romania / Hybrid, October 6–7 (2022). https://doi.org/10.31234/osf.io/gptvz

  37. H.A. Kang, A. Sales, T.A. Whittaker, Flow with an intelligent tutor: A latent variable modeling approach to tracking flow during artificial tutoring. Behav. Res. Methods 1–24 (2023). https://doi.org/10.3758/s13428-022-02041-w

  38. Y.G. Kang, H.D. Song, H. Yun, Y. Jo, The effect of virtual reality media characteristics on flow and learning transfer in job training: The moderating effect of presence. J. Comput. Assist. Learn. 38(6), 1674–1685 (2022). https://doi.org/10.1111/jcal.12702

    Article  Google Scholar 

  39. J. Keller, H. Bless, Flow and regulatory compatibility: an experimental approach to the flow model of intrinsic motivation. Pers. Soc. Psychol. Bull. 34(2), 196–209 (2008). https://doi.org/10.1177/0146167207310026

    Article  Google Scholar 

  40. J. Keller, H. Bless, F. Blomann, D. Kleinböhl, Physiological aspects of flow experiences: Skills-demand-compatibility effects on heart rate variability and salivary cortisol. J. Exp. Soc. Psychol. 47(4), 849–852 (2011). https://doi.org/10.1016/j.jesp.2011.02.004

    Article  Google Scholar 

  41. D. Kirsh, P. Maglio, On distinguishing epistemic action from pragmatic action. J. Cogn. Sci. 18(4), 513–549 (1994). https://doi.org/10.1016/0364-0213(94)90007-8

    Article  Google Scholar 

  42. J.M. Kivikangas, Psychophysiology of flow experience: An explorative study. (Master’s Thesis, University of Helsinki, Finland, 2006). http://ethesis.helsinki.fi/julkaisut/kay/psyko/pg/kivikangas/

  43. C.R. Landsberg, R.S. Astwood Jr., W.L. Van Buskirk, L.N. Townsend, N.B. Steinhauser, A.D. Mercado, Review of adaptive training system techniques. Mil. Psychol. 24(2), 96–113 (2012). https://doi.org/10.1080/08995605.2012.672903

    Article  Google Scholar 

  44. J.K. Lindstedt, W.D. Gray, Meta-T: TetrisⓇ as an experimental paradigm for cognitive skills research. Behav. Res. Methods 47(4), 945–965 (2015). https://doi.org/10.3758/s13428-014-0547-y

    Article  Google Scholar 

  45. T. Lynch, I. Ghergulescu, Large scale evaluation of learning flow, in 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT) (IEEE, 2017), pp. 62–64. https://doi.org/10.1109/ICALT.2017.98

  46. M. Mauri, P. Cipresso, A. Balgera, M. Villamira, G. Riva, Why is Facebook so successful? Psychophysiological measures describe a core flow state while using Facebook. Cyberpsychol. Behav. Soc. Netw. 14(12), 723–731 (2011). https://doi.org/10.1089/cyber.2010.0377

    Article  Google Scholar 

  47. S. Miranda, I. Trigo, R. Rodrigues, M. Duarte, Addiction to social networking sites: motivations, flow, and sense of belonging at the root of addiction. Technol. Forecast. Soc. Change 188. (2023). (Elsevier)

    Google Scholar 

  48. H. Monkaresi, N. Bosch, R.A. Calvo, S.K. D’Mello, Automated detection of engagement using video-based estimation of facial expressions and heart rate. IEEE Trans. Affect. Comput. 8(1), 15–28 (2016). https://doi.org/10.1109/TAFFC.2016.2515084

    Article  Google Scholar 

  49. M. Mulders, Experiencing flow in virtual reality: an investigation of complex interaction structures of learning-related variables, in 19th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2022). (2022)

    Google Scholar 

  50. J. Nakamura, M. Csikszentmihalyi, Flow theory and research, in The Oxford Handbook of Positive Psychology. ed. by S.J. Lopez, C.R. Snyder (Oxford University Press, 2009), pp.195–206

    Google Scholar 

  51. Y.I. Nakano, R. Ishii, Estimating user's engagement from eye-gaze behaviors in human-agent conversations, in Proceedings of the 15th International Conference on Intelligent user Interfaces (Feb 2010), pp. 139–148. https://doi.org/10.1145/1719970.1719990

  52. M. Neta, C.J. Norris, P.J. Whalen, Corrugator muscle responses are associated with individual differences in positivity-negativity bias. Emotion 9(5), 640 (2009). https://doi.org/10.1037/a0016819

    Article  Google Scholar 

  53. M. Ninaus, S. Greipl, K. Kiili, A. Lindstedt, S. Huber, E. Klein, H.O. Karnath, K. Moeller, Increased emotional engagement in game-based learning–A machine learning approach on facial emotion detection data. Comput. Educ. 142, 103641 (2019). https://doi.org/10.1016/j.compedu.2019.103641

    Article  Google Scholar 

  54. K. J. O'Neill, Induction and Transferal of Flow in the Game Tetris. Master's thesis, Wright State University (OhioLINK Electronic Theses and Dissertations Center, 2020)

    Google Scholar 

  55. L. Okagaki, P.A. Frensch, Effects of video game playing on measures of spatial performance: gender effects in late adolescence. J. Appl. Dev. Psychol. 15(1), 33–58 (1994). https://doi.org/10.1016/0193-3973(94)90005-1

    Article  Google Scholar 

  56. J. Palomäki, T. Tammi, N. Lehtonen, N. Seittenranta, M. Laakasuo, S. Abuhamdeh, O. Lappi, B.U. Cowley, The link between flow and performance is moderated by task experience. Comput. Hum. Behav. 124, Article 106891 (2021). https://doi.org/10.1016/j.chb.2021.106891

  57. C. Peifer, J. Tan, The psychophysiology of flow experience, in Advances in Flow Research, ed. by C. Peifer, S. Engeser (Springer, 2021), pp. 191–230. https://doi.org/10.1007/978-3-030-53468-4_8

  58. C. Peifer, A. Pollak, O. Flak, A. Pyszka, M.A. Nisar, M.T. Irshad, M. Grzegorzek, B. Kordyaka, B. Kożusznik, The symphony of team flow in virtual teams: using artificial intelligence for its recognition and promotion. Front. Psychol. 12 (2021). https://doi.org/10.3389/fpsyg.2021.697093

  59. C. Peifer, H. Schächinger, S. Engeser, C.H. Antoni, Cortisol effects on flow-experience. Psychopharmacology 232(6), 1165–1173 (2015). https://doi.org/10.1007/s00213-014-3753-5

    Article  Google Scholar 

  60. D.N. Perkins, G. Salomon, Transfer of Learning, in The International Encyclopedia of Education, 2nd edn., ed. by T. Husén, T.N. Postlethwaite (Pergamon, Oxford, 1992), pp.425–441

    Google Scholar 

  61. K. Procci, N. James, C. Bowers, The effects of gender, age, and experience on game engagement, in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, No. 1. (Los Angeles, CA: SAGE Publications, Sage CA, Sept 2013), pp. 2132–2136. https://doi.org/10.1177/1541931213571475

  62. A. Psaltis, K.C. Apostolakis, K. Dimitropoulos, P. Daras, Multimodal student engagement recognition in prosocial games. IEEE Trans. Games 10(3), 292–303 (2017). https://doi.org/10.1109/TCIAIG.2017.2743341

    Article  Google Scholar 

  63. F.L. Schmidt, L.B. Kaplan, Composite vs multiple criteria: a review and resolution of the controversy. Pers. Psychol. 24(3), 419–434 (1971). https://doi.org/10.1111/j.1744-6570.1971.tb00365.x

    Article  Google Scholar 

  64. M. Shehata, M. Cheng, A. Leung, N. Tsuchiya, D.A. Wu, C.H. Tseng, S. Nakauchi, S., Shimojo, Team flow is a unique brain state associated with enhanced information integration and interbrain synchrony. eNeuro, 8(5) (2021). ENEURO.0133–21.2021. https://doi.org/10.1523/ENEURO.0133-21.2021

  65. M. Shehata, S. Elnagar, S. Yasunaga, S. Nakauchi, S. Shimojo, Flow of the eye: gaze direction as an objective measure of flow experience. J. Vis. 18(10), 1205–1205 (2018). https://doi.org/10.1167/18.10.1205

    Article  Google Scholar 

  66. C. Sibert, W.D. Gray, J.K. Lindstedt, Interrogating feature learning models to discover insights into the development of human expertise in a real-time, dynamic decision-making task. Top. Cogn. Sci. 9(2), 374–394 (2017). https://doi.org/10.1111/tops.12225

    Article  Google Scholar 

  67. R.D. Spain, H.A. Priest, J.S. Murphy, Current trends in adaptive training with military applications: an introduction. Mil. Psychol. 24(2), 87–95 (2012). https://doi.org/10.1080/08995605.2012.676984

    Article  Google Scholar 

  68. A.A. Stone, S. Shiffman, Ecological momentary assessment (EMA) in behavioral medicine. Ann. Behav. Med. 16(3), 199–202 (1994). https://doi.org/10.1093/abm/16.3.199

    Article  Google Scholar 

  69. S. Triberti, A. Di Natale, A. Gaggioli, Flowing technologies: the role of flow and related constructs in human-computer interaction, in Advances in Flow Research ed. by C. Peifer, S. Engeser (Springer, 2021), pp. 393–416. https://doi.org/10.1007/978-3-030-53468-4_15 

  70. T.J. Trull, U. Ebner-Priemer, The role of ambulatory assessment in psychological science. Curr. Dir. Psychol. Sci. 23(6), 466–470 (2014). https://doi.org/10.1177/0963721414550706

    Article  Google Scholar 

  71. H. Tseng, X. Yi, B. Cunningham, Learning technology acceptance and continuance intention among business students: the mediating effects of confirmation, flow, and engagement. Aust. J. Educ. Technol. 38(3), 70–86 (2022). https://doi.org/10.14742/ajet.7219 

  72. J.J.J. van den Hout, Team flow: from concept to application. (Phd Thesis 1 (Research TU/e / Graduation TU/e), Industrial Engineering and Innovation Sciences). Technische Universiteit Eindhoven (2016)

    Google Scholar 

  73. J.J.J. van den Hout, O.C. Davis, M.C.D.P. Weggeman, The conceptualization of team flow. J. Psychol. 152(6), 388–423 (2018). https://doi.org/10.1080/00223980.2018.1449729

    Article  Google Scholar 

  74. G. Wang, Humans in the loop: the design of interactive AI systems. J. Artif. Intell. Res. 64, 243–252 (2019). https://doi.org/10.1613/jair.1.11345

    Article  Google Scholar 

  75. J. Whitehill, Z. Serpell, Y.C. Lin, A. Foster, J.R. Movellan, The faces of engagement: automatic recognition of student engagement from facial expressions. IEEE Trans. Affect. Comput. 5(1), 86–98 (2014). https://doi.org/10.1109/TAFFC.2014.2316163

    Article  Google Scholar 

Download references

Acknowledgements

IJ’s contribution to this work was supported in part by The Office of Naval Research grant number N00014-16-l-2047 P00004 to Brandon Minnery, Ion Juvina, and Assaf Harel.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ion Juvina .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Juvina, I. et al. (2024). Human-AI Coordination to Induce Flow in Adaptive Learning Systems. In: Kolski, C., Mihăescu, M.C., Rebedea, T. (eds) AI Approaches for Designing and Evaluating Interactive Intelligent Systems. ROCHI 2022. Learning and Analytics in Intelligent Systems, vol 36. Springer, Cham. https://doi.org/10.1007/978-3-031-53957-2_7

Download citation

Publish with us

Policies and ethics