Abstract
Game-based learning environments create rich learning experiences that are both effective and engaging. Recent years have seen growing interest in data-driven techniques for tutorial planning, which dynamically personalize learning experiences by providing hints, feedback, and problem scenarios at run-time. In game-based learning environments, tutorial planners are designed to adapt gameplay events in order to achieve multiple objectives, such as enhancing student learning or student engagement, which may be complementary or competing aims. In this paper, we introduce a multi-objective reinforcement learning framework for inducing game-based tutorial planners that balance between improving learning and engagement in game-based learning environments. We investigate a model-based, linear-scalarized multi-policy algorithm, Convex Hull Value Iteration, to induce a tutorial planner from a corpus of student interactions with a game-based learning environment for middle school science education. Results indicate that multi-objective reinforcement learning creates policies that are more effective at balancing multiple reward sources than single-objective techniques. A qualitative analysis of select policies and multi-objective preference vectors shows how a multi-objective reinforcement learning framework shapes the selection of tutorial actions during students’ game-based learning experiences to effectively achieve targeted learning and engagement outcomes.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adams, D.M., Mayer, R.E., MacNamara, A., Koenig, A., Wainess, R.: Narrative games for learning: testing the discovery and narrative hypotheses. J. Educ. Psychol. 104(1), 235–249 (2012)
Barrett, L., Narayanan, S.: Learning all optimal policies with multiple criteria. In: Proceedings of the 25th International Conference on Machine Learning, pp. 41–47. ACM (2008)
Chi, M., VanLehn, K., Litman, D., Jordan, P.: Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies. User Model. User-Adap. Inter. 21(1–2), 137–180 (2011)
Lee, S., Rowe, J., Mott, B., Lester, J.: A supervised learning framework for modeling director agent strategies in educational interactive narrative. IEEE Trans. Comput. Intell. AI Games 6(2), 203–215 (2014)
Nelson, M., Roberts, D., Isbell, C., Mateas, M.: Reinforcement learning for declarative optimization-based drama management. In: Proceedings of the 5th International Conference on Autonomous Agents and Multiagent Systems, pp. 775–782. ACM, Japan (2006)
Prensky, M.: Digital Game-based Learning. McGraw-Hill, New York (2001)
Roijers, D., Vamplew, P., Whiteson, S., Dazeley, R.: A survey of multi-objective sequential decision-making. J. Artif. Intell. Res. 48, 67–113 (2013)
Rowe, J.: Narrative-centered tutorial planning with concurrent markov decision processes. Ph.D. diss., Dept. of Computer Science, North Carolina State University (2013)
Rowe, J., Mott, B., Lester, J.: Optimizing player experience in interactive narrative planning: a modular reinforcement learning approach. In: Proceedings of the 10th Artificial Intelligence and Interactive Digital Entertainment Conference, pp. 160–166. Raleigh, NC (2014)
Rowe, J., Lester, J.: Improving student problem solving in narrative-centered learning environments: a modular framework. In: Proceedings of the 17th International Conference on Artificial Intelligence in Education, pp. 419–428. Madrid, Spain (2015)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Wiering, M., Withagen, M., Drugan, M.: Model-based multi-objective reinforcement learning. In: IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 1–6 (2014)
Witmer, B., Singer, M.: Measuring presence in virtual environments: a presence questionnaire. Presence Teleoperators Virtual Environ. 7(3), 225–240 (1998)
Clark, D.B., Tanner-Smith, E., Killingsworth, S.: Digital games, design, and learning: a systematic review and meta-analysis. Rev. Educ. Res. 86(1), 79–122 (2015)
Wouters, P., van Nimwegen, C., van Oostendorp, H., van der Spek, E.D.: A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 105, 249–265 (2013)
Mandel, T., Liu, Y., Levine, S., Brunskill, E., Popovic, Z.: Offline policy evaluation across representations with applications to educational games. In: Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems, pp. 1077–1084. Richland, SC (2014)
Castelletti, A., Pianosi, F., Restelli, M.: A multiobjective reinforcement learning approach to water resources systems operation: pareto frontier approximation in a single run. Water Resour. Res. 49(6), 3476–3486 (2013)
Houli, D., Zhiheng, L., Yi, Z.: Multiobjective reinforcement learning for traffic signal control using vehicular ad hoc network. EURASIP J. Adv. Signal Process. 1, 724035 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Sawyer, R., Rowe, J., Lester, J. (2017). Balancing Learning and Engagement in Game-Based Learning Environments with Multi-objective Reinforcement Learning. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-61425-0_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-61424-3
Online ISBN: 978-3-319-61425-0
eBook Packages: Computer ScienceComputer Science (R0)