Erik Harpstead
Christopher J. MacLellan
Brad A. Myers
ABSTRACT
As educational games have become a larger field of study, there has been a growing need for analytic methods that can be used to assess game design and inform iteration. While much previous work has focused on the measurement of student engagement or learning at a gross level, we argue that new methods are necessary for measuring the alignment of a game to its target learning goals at an appropriate level of detail to inform design decisions. We present a novel technique that we have employed to examine alignment in an open-ended educational game. The approach is based on examining how the game reacts to representative student solutions that do and do not obey target principles. We demonstrate this method using real student data and discuss how redesign might be informed by these techniques.
Supplemental Material
- Aleven, V., Myers, E., Easterday, M., and Ogan, A. Toward a Framework for the Analysis and Design of Educational Games. Proc. DiGITEL 2010, IEEE (2010), 69--76. Google Scholar
Digital Library
- Alhabash, S. and Wise, K. PeaceMaker: Changing Students' Attitudes Toward Palestinians and Israelis Through Video Game Play. International Journal of Communication 6, (2012), 356--380.Google Scholar
- Andersen, E., Liu, Y., Apter, E., Boucher-genesse, F., and Popovi, Z. Gameplay Analysis through State Projection. Proc. FDG 2010, ACM Press (2010). Google ScholarDigital Library
- Andersen, E., Liu, Y., Snider, R., Szeto, R., Cooper, S., and Popovi, Z. On the Harmfulness of Secondary Game Objectives. Proc. FDG 2011, ACM Press (2011). Google ScholarDigital Library
- Christel, M.G., Stevens, S.M., Maher, B.S., et al. RumbleBlocks: Teaching Science Concepts to Young Children through a Unity Game. Proc. CGames 2012, IEEE (2012), 162--166. Google ScholarDigital Library
- Clark, D.B., Tanner-Smith, E.E., and Killingsworth, S. Digital Games for Learning: A Systematic Review and Meta-Analysis (Executive Summary). Menlo Park, CA, 2013.Google Scholar
- Gee, J.P. What video games have to teach us about learning and literacy. Palgrave Macmillan, New York, 2003. Google ScholarDigital Library
- Harpstead, E., Maclellan, C.J., Koedinger, K.R., Aleven, V., Dow, S.P., and Myers, B.A. Investigating the Solution Space of an Open-Ended Educational Game Using Conceptual Feature Extraction. Proc. EDM 2013, (2013).Google Scholar
- Harpstead, E., Myers, B.A., and Aleven, V. In Search of Learning: Facilitating Data Analysis in Educational Games. Proc. CHI 2013, ACM Press (2013), 79. Google ScholarDigital Library
- Hunicke, R., Leblanc, M., and Zubek, R. MDA : A Formal Approach to Game Design and Game Research. Proc. of the AAAI Workshop on Challenges in Game AI, (2004), 1--5.Google Scholar
- Isbister, K., Flanagan, M., and Hash, C. Designing Games for Learning?: Insights from Conversations with Designers. Proc. CHI 2010, ACM Press (2010), 2041--2044. Google ScholarDigital Library
- Johnson, M.W., Eagle, M., and Barnes, T. InVis : An Interactive Visualization Tool for Exploring Interaction Networks. Proc. EDM 2013, (2013), 65.Google Scholar
- Khaled, R. and Ingram, G. Tales from the Front Lines of a Large-Scale Serious Game Project. Proc. CHI 2012, ACM Press (2012), 69--78. Google ScholarDigital Library
- Koedinger, K.R., Corbett, A.T., and Perfetti, C. The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning. Cognitive Science 36, 5 (2012), 757--98.Google ScholarCross Ref
- Koedinger, K.R. and Nathan, M.J. The Real Story Behind Story Problems: Effects of Representations on Quantitative Reasoning. The Journal of the Learning Sciences 13, 2 (2004), 129--164.Google ScholarCross Ref
- Kohavi, R., Deng, A., Frasca, B., Longbotham, R., Walker, T., and Xu, Y. Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained. Proc. KDD 2012, ACM Press (2012), 786--794. Google ScholarDigital Library
- Kohavi, R., Henne, R.M., and Sommerfield, D. Practical Guide to Controlled Experiments on the Web : Listen to Your Customers not to the HiPPO. Proc. KDD 2007, ACM Press (2007), 1--9. Google ScholarDigital Library
- Leighton, J.P. and Gokiert, R.J. The Cognitive Effects of Test Item Features: Informing Item Generation by Identifying Construct Irrelevant Variance. Proc. NCME 2005, (2005), 1--26.Google Scholar
- Linehan, C., Kirman, B., Lawson, S., and Chan, G.G. Practical, Appropriate, Empirically-Validated Guidelines for Designing Educational Games. Proc. CHI 2011, ACM Press (2011), 1979--1988. Google ScholarDigital Library
- Lomas, D., Forlizzi, J.L., Koedinger, K.R., and Patel, K. Optimizing Challenge in an Educational Game Using Large-Scale Design Experiments. Proc. CHI 2013, (2013), 89--98. Google ScholarDigital Library
- Maciuszek, D. and Martens, A. Patterns for the design of educational games. In F. Edvarsen and H. Kulle, eds., Educational Games Design Learning and Applications. Nova publishers, 2010, 263--279.Google Scholar
- Mayer, R.E. and Moreno, R. Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educational Psychologist 38, 1 (2003), 43--52.Google ScholarCross Ref
- Mccoy, J., Treanor, M., Samuel, B., Reed, A.A., Wardrip-fruin, N., and Mateas, M. Prom Week. Proc. FDG 2012, ACM Press (2012), 235--237. Google ScholarDigital Library
- Miller, C.S., Lehman, J.F., and Koedinger, K.R. Goals and learning in microworlds. Cognitive Science 23, 3 (1999), 305--336.Google ScholarCross Ref
- Nacke, L.E., Drachen, A., and Göbel, S. Methods for Evaluating Gameplay Experience in a Serious Gaming Context. Journal of Computer Science in Sport 9, 2 (2010), 1--12.Google Scholar
- Nathan, M.J. Knowledge and Situational Feedback in a Learning Environment for Algebra Story Problems. Interactive Learning Environments 5, 1 (1998), 134--159.Google ScholarCross Ref
- Nielsen, J. and Molich, R. Heuristic Evaluation of User Interfaces. Proc. CHI 1990, ACM Press (1990), 249--256. Google ScholarDigital Library
- Pashler, H., Bain, P.M., Bottge, B.A., et al. Organizing Instruction and Study to Improve Student Learning (NCER 2007--2004). Washington, D.C., 2007.Google Scholar
- Putnam, R.D. Bowling Alone: America's Declining Social Capital. Journal of Democracy 6, 1 (1995), 65--78.Google ScholarCross Ref
- Rau, M.A., Aleven, V., and Rohrbach, S. Why Interactive Learning Environments Can Have It All?: Resolving Design Conflicts Between Competing Goals. Proc. CHI 2013, ACM Press (2013), 109--118. Google ScholarDigital Library
- Seif El-Nasr, M., Drachen, A., and Canossa, A., eds. Game Analytics. Springer London, London, 2013.Google ScholarCross Ref
- Wiggins, G. and McTighe, J. Understanding by Design. Pearson, 2005.Google Scholar
Index Terms
- Using extracted features to inform alignment-driven design ideas in an educational game
Recommendations
The Eduventure II. An Approach to Educational Game Design
CW '08: Proceedings of the 2008 International Conference on CyberworldsVideo games engage players and can train formal skills. However, the potential of educational games for knowledge acquisition lacks research. Moreover, many educational games suffer from low-grade design, therefore impeding the flow of the game and ...
Data-Driven Game Design: The Case of Difficulty in Educational Games
Addressing Global Challenges and Quality EducationAbstractThere is increasing interest in using data to design digital games that serve the purposes of learning and assessment. One game element, difficulty, could benefit vastly from applying data-driven methods as it affects both players’ overall ...
Using Empirical Learning Curve Analysis to Inform Design in an Educational Game
CHI PLAY '15: Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in PlayHaving insights into players' learning has important implications for design in an educational game. Empirical learning curve analysis is an approach from intelligent tutoring systems literature for measuring student learning within a system in terms of ...
Comments