Skip to main content

Advertisement

SpringerLink
Log in

What We Can Learn from the Data: A Multiple-Case Study Examining Behavior Patterns by Students with Different Characteristics in Using a Serious Game

  • Published:
Technology, Knowledge and Learning Aims and scope Submit manuscript

Abstract

Using a multi-case approach, we examined students’ behavior patterns in interacting with a serious game environment using the emerging technologies of learning analytics and data visualization in order to understand how the patterns may vary according to students’ learning characteristics. The results confirmed some preliminary findings from our previous research, but also revealed patterns that would not be easily detected without data visualizations. Such findings provided insights about designing effective learning scaffolds to support the development of problem-solving skills in young learners and will guide our next-step research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Andersen, E., Liu, Y. E., Apter, E., Boucher-Genesse, F., & Popović, Z. (2010). Gameplay analysis through state projection. In Proceedings of the fifth international conference on the foundations of digital games (pp. 1–8). USA. doi:10.1145/1822348.1822349

  • Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., et al. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.

    Google Scholar 

  • Asgari, M., & Kaufman, D. (2010). Does fantasy enhance learning in digital games? In D. Kaufman & L. Sauve (Eds.), Educational gameplay and simulation environments: Case studies and lessons learned (pp. 84–95). Hershey, PA: IGI Global.

  • Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics. Washington, DC: U.S. Department of Education. Retrieved from http://www.ed.gov/edblogs/technology/files/2012/03/edm-la-brief.pdf

  • Blikstein, P. (2011). Using learning analytics to assess students’ behavior in open-ended program-ming tasks. In P. Long, G. Siemens, G. Conole, & D. Gašević (Eds.), Proceedings of the learning analytics knowledge conference (pp. 110–116). New York, NY: ACM.

    Chapter  Google Scholar 

  • Bogard, T., Liu, M., & Chiang, Y. H. (2013). Thresholds of knowledge development in complex problem solving: A multiple-case study of advanced learners’ cognitive processes. Educational Technology Research and Development, 61(3), 465–503. doi:10.1007/s11423-013-9295-4.

    Article  Google Scholar 

  • Bransford, J. D., & Stein, B. S. (1984). The IDEAL problem solver. New York, NY: W. H. Feeman and Company.

    Google Scholar 

  • Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., & Pidruzny, J. N. (2009). The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social Psychology, 45(4), 624–634.

    Article  Google Scholar 

  • Buckingham Shum, S. (2012). Learning analytics. UNESCO policy brief. Retrieved from http://iite.unesco.org/pics/publications/en/files/3214711.pdf

  • Chittaro, L., & Ieronutti, L. (2004). A visual tool for tracing users’ behavior in virtual environments. In Proceedings of the working conference on Advanced visual interfaces (pp. 40–47). USA.

  • Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4), 715.

    Article  Google Scholar 

  • Dixit, P. N., & Youngblood, G. M. (2008) Understanding playtest data through visual data mining in interactive 3D environments. Paper presented at the 12th international conference on computer games: AI, Animation, Mobile, Interactive Multimedia and Serious Games (CGAMES). Louisville, KY.

  • Drachen, A., & Canossa, A. (2009a). Analyzing spatial user behavior in computer games using geographic information systems. In Proceedings of the 13th international MindTrek conference: Everyday life in the ubiquitous era (pp. 182–189). New York, NY: ACM. doi:10.1145/1621841.1621875

  • Drachen, A., & Canossa, A. (2009b). Towards gameplay analysis via gameplay metrics. In Proceedings of the 13th international MindTrek conference: Everyday life in the ubiquitous era (pp. 202–209). New York, NY: ACM. doi:10.1145/1621841.1621878

  • Dreyfus, S. E. (2004). The five-stage model of adult skill acquisition. Bulletin of Science Technology Society, 24(3), 177–181. doi:10.1177/0270467604264992.

    Article  Google Scholar 

  • Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman, R. R. (2006). The Cambridge handbook of expertise and expert performance. New York: Cambridge University Press.

    Book  Google Scholar 

  • Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441–467. doi:10.1177/1046878102238607.

    Article  Google Scholar 

  • Hsieh, P., Cho, Y., Liu, M., & Schallert, D. (2008). Examining the interplay between middle school students’ achievement goals and self-efficacy in a technology-enhanced learning environment. American Secondary Education, 36(3), 33–50.

    Google Scholar 

  • Hwang, W., Shadiev, R., Wang, C., & Huang, Z. (2011). A pilot study of cooperative programming learning behavior and its relationship with students’ learning performance. Computers & Education, 58, 1267–1281.

    Article  Google Scholar 

  • Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2014). The NMC horizon report: 2014 higher education edition. Austin, TX: The New Media Consortium.

    Google Scholar 

  • Johnson, L., Adams Becker, S., Estrada, V., & Freeman, A. (2015). NMC horizon report: 2015 higher (Education ed.). Austin, TX: The New Media Consortium.

    Google Scholar 

  • Koedinger, K., Cunningham, K., Skogsholm, A., & Leber, B. (2008). An open repository and analysis tools for fine-grained, longitudinal learner data. In Proceedings of first international conference on educational data mining (pp. 157–166). Canada.

  • Linek, S. B., Öttl, G., & Albert, D. (2010). Non-invasive data tracking in educational games: Combination of logfiles and natural language processing. In Proceedings of the international technology, education and development conference (INTED).

  • Liu, M., & Bera, S. (2005). An analysis of cognitive tool use patters in hypermedia learning environment. Educational Technology Research and Development, 53(1), 5–21.

    Article  Google Scholar 

  • Liu, M., Bera, S., Corliss, S., Svinicki, M., & Beth, A. (2004). Understanding the connection between cognitive tool use and cognitive processes as used by sixth graders in a problem-based hypermedia learning environment. Journal of Educational Computing Research, 31(3), 309–334.

    Article  Google Scholar 

  • Liu, M., Horton, L., Corliss, S. B., Svinicki, M. D., Bogard, T., Kim, J., & Chang, M. (2009). Students’ problem solving as mediated by their cognitive tool use: A study of tool use patterns. Journal of Educational Computing Research, 40(1), 111–139.

    Article  Google Scholar 

  • Liu, M., Horton, L., Kang, J., Kimmons, R., & Lee, J. (2013a). Using a ludic simulation to make learning of middle school space science fun. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 5(1), 66–86.

    Article  Google Scholar 

  • Liu, M., Kang, J., Lee, J., Winzeler, E., & Liu, S. (2015). Examining through visualization what tools learners access as they play a serious game for middle school science. In C. S. Loh, Y. Sheng, & D. Ifenthaler (Eds.), Serious games analytics: Methodologies for performance measurement, assessment, and improvement (pp. 181–208). Switzerland: Springer. doi:10.1007/978-3-319-05834-4.

    Chapter  Google Scholar 

  • Liu, M., Rosenblum, J. A., Horton, L., & Kang, J. (2014). Designing science learning with game-based approaches. Computers in the Schools, 31(1–2), 84–102. doi:10.1080/07380569.2014.879776.

    Article  Google Scholar 

  • Liu, M., Yuen, T. T., Horton, L., Lee, J., Toprac, P., & Bogard, T. (2013b). Designing technology-enriched cognitive tools to support young learners’ problem solving. The International Journal of Cognitive Technology, 18(1), 14–21.

    Google Scholar 

  • Loh, C. S., & Sheng, Y. (2013). Measuring the (dis-)similarity between expert and novice behaviors as serious games analytics. Education and Information Technologies,. doi:10.1007/s10639-013-9263-y.

    Google Scholar 

  • Lynn, S. J., & Rhue, J. W. (1986). The fantasy-prone person: Hypnosis, imagination, and creativity. Journal of Personality and Social Psychology, 51(2), 404–408.

    Article  Google Scholar 

  • Malone, T. W. (1981). Toward a theory of intrinsically motivating instruction. Cognitive Science, 5(4), 333–369. doi:10.1207/s15516709cog0504_2.

    Article  Google Scholar 

  • Malone, T. W., & Lepper, M. R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. Aptitude, Learning, and Instruction, 3, 223–253.

    Google Scholar 

  • Martin, T., & Sherin, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: An introduction to the special issue. Journal of the Learning Sciences, 22(4), 511–520. doi:10.1080/10508406.2013.840466.

    Article  Google Scholar 

  • Merckelbach, H., Horselenberg, R., & Muris, P. (2001). The Creative Experiences Questionnaire (CEQ): A brief self-report measure of fantasy proneness. Personality and Individual Differences, 31(6), 987–995. doi:10.1016/S0191-8869(00)00201-4.

    Article  Google Scholar 

  • Moura, D., Seif El-Nasr, M., & Shaw, C. C. (2011). Visualizing and understanding players’ behavior in video games: Discovering patterns and supporting aggregation and comparison. In Proceedings of the 2011 ACM SIGGRAPH symposium on video games (pp. 11–15). New York, NY: ACM Press.

  • Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12–27. doi:10.1002/widm.1075.

    Google Scholar 

  • Scarlatos, L. L., & Scarlatos, T. (2010). Visualizations for the assessment of learning in computer games. In Paper presented at the 7th international conference & expo on emerging technologies for a smarter world (CEWIT 2010), Incheon, Korea.

  • Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400. doi:10.1177/0002764213498851.

    Article  Google Scholar 

  • Siemens, G., & d Baker, R. S. J. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252–254). New York, NY: ACM.

  • Stake, R. E. (2005). Multiple case study analysis. New York, NY: Guilford Press.

    Google Scholar 

  • Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (2012). Dataset-driven research to support learning and knowledge analytics. Educational Technology and Society, 15(3), 133–148.

    Google Scholar 

  • Wallner, G., & Kriglstein, S. (2013). Visualization-based analysis of gameplay data—A review of literature. Entertainment Computing, 4(3), 143–155. doi:10.1016/j.entcom.2013.02.002.

    Article  Google Scholar 

  • Wiest, L. R. (2001). The role of fantasy contexts in word problems. Mathematics Education Research Journal, 13(2), 74–90.

    Article  Google Scholar 

  • Wilson, K. A., Bedwell, W. L., Lazzara, E. H., Salas, E., Burke, C. S., Estock, J. L., & Conkey, C. (2009). Relationships between game attributes and learning outcomes: Review and research proposals. Simulation & Gaming, 40(2), 217–266. doi:10.1177/1046878108321866.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Learning Technologies Program, The University of Texas at Austin, Austin, TX, USA

    Min Liu, Jina Kang & Sa Liu

  2. BK21Plus Institute of Future Education Design, Seoul National University, Seoul, South Korea

    Jaejin Lee

Authors
  1. Min Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaejin Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jina Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sa Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, M., Lee, J., Kang, J. et al. What We Can Learn from the Data: A Multiple-Case Study Examining Behavior Patterns by Students with Different Characteristics in Using a Serious Game. Tech Know Learn 21, 33–57 (2016). https://doi.org/10.1007/s10758-015-9263-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10758-015-9263-7

Keywords

Search

Navigation