Skip to main content
SpringerLink
Log in

Open Game Data: A Technical Infrastructure for Open Science with Educational Games

  • Conference paper
  • First Online:
Serious Games (JCSG 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14309))

Included in the following conference series:

Abstract

In this paper we describe a technical infrastructure, entitled Open Game Data, for conducting educational game research using open science, educational data mining and learning engineering approaches. We describe a modular data pipeline which begins with telemetry events from gameplay and ends with real time APIs and automated archival exports that support research. We demonstrate the usefulness of this infrastructure by summarizing several game research projects that have utilized and contributed back to Open Game Data. We then conclude with current efforts to expand the infrastructure.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Advanced Distributed Learning: Experience API (xAPI) Standard (n.d). https://adlnet.gov/projects/xapi. Accessed 30 July 2023

  2. Baker, R.: Modeling and understanding students’ off-task behavior in intelligent tutoring systems. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI 2007, pp. 1059–1068 (2007). https://doi.org/10.1145/1240624.1240785

  3. Baker, R.S., Boser, U.: High-leverage opportunities for learning engineering, p. 48. Penn Center for Learning Analytics (2022). https://learninganalytics.upenn.edu/Learning_Engineering_recommendations.pdf

  4. Clark, D.B., Tanner-Smith, E.E., Killingsworth, S.S.: Digital games, design, and learning: a systematic review and meta-analysis. Rev. Educ. Res. 86(1), 79–122 (2016). https://doi.org/10.3102/0034654315582065

    Article  Google Scholar 

  5. D’Angelo, C., Rutstein, D., Harris, C., Haertel, G., Bernard, R., Borokhovski, E.: Simulations for STEM learning: systematic review and meta-analysis. SRI Int. 5, 1–5 (2014)

    Google Scholar 

  6. DiCerbo, K.E., Kidwai, K.: Detecting player goals from game log files. In: Proceedings of the 6th International Conference on Educational Data Mining (EDM 2013), p. 2 (2013)

    Google Scholar 

  7. Gagnon, D.J., et al.: Exploring players’ experience of humor and snark in a grade 3–6 history practices game. In: GLS 13.0 Conference Proceedings. Games, Learning Society, Irving, CA (2022)

    Google Scholar 

  8. Gagnon, D.J., Harpstead, E., Slater, S.: Comparison of off the shelf data mining methodologies in educational game analytics. In: Joint Proceedings of the Workshops of the 12th International Conference on Educational Data Mining Co-Located with the 12th International Conference on Educational Data Mining (EDM 2019), pp. 38–43 (2019)

    Google Scholar 

  9. Gee, J. P.: What Video Games Have to Teach us About Learning and Literacy (1. Paperback ed). Palgrave Macmillan (2004)

    Google Scholar 

  10. Holstein, K., McLaren, B.M., Aleven, V.: Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity. J. Learn. Anal. 6(2), 27–52 (2019). https://doi.org/10.18608/jla.2019.62.3

    Article  Google Scholar 

  11. Ioannidis, J.P.A.: Why most published research findings are false. PLoS Med. 2(8), e124 (2005). https://doi.org/10.1371/journal.pmed.0020124

    Article  Google Scholar 

  12. Kai, S., Almeda, M.V., Baker, R.S., Heffernan, C., Heffernan, N.: Decision tree modeling of wheel-spinning and productive persistence in skill builders. JEDM | J. Educ. Data Min. 10(1), 36–71 (2018)

    Google Scholar 

  13. Kamarainen, A.M., et al.: EcoMOBILE: integrating augmented reality and probeware with environmental education field trips. Comput. Educ. 68, 1–12 (2013). https://doi.org/10.1016/j.compedu.2013.02.018

    Article  Google Scholar 

  14. Kim, Y.J., Metcalf, S.J., Scianna, J., Perez, G., Gagnon, D.J.: AquaLab: establishing validity of an adventure game for middle school science practices. In: Iyer, S. (ed.) Proceedings of the 30th International Conference on Computers in Education (2022)

    Google Scholar 

  15. Kim, Y.J., Shute, V.J.: The interplay of game elements with psychometric qualities, learning, and enjoyment in game-based assessment. Comput. Educ. 87, 340–356 (2015). https://doi.org/10.1016/j.compedu.2015.07.009

    Article  Google Scholar 

  16. Koedinger, K.R., Baker, R.S.J.D., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J.: A data repository for the EDM community: the PSLC DataShop. In: Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S. (eds.) Handbook of Educational Data Mining. CRC Press, Boca Raton (2010)

    Google Scholar 

  17. Kraker, P., Leony, D., Reinhardt, W., Gü, N.A., Beham, N.: The case for an open science in technology enhanced learning. Int. J. Technol. Enhanc. Learn. 3(6), 643 (2011). https://doi.org/10.1504/IJTEL.2011.045454

    Article  Google Scholar 

  18. Lomas, D.: Optimizing challenge in an educational game using large-scale design experiments, pp. 1–10 (2013)

    Google Scholar 

  19. Makel, M.C., Plucker, J.A.: Facts are more important than novelty: replication in the education sciences. Educ. Res. 43(6), 304–316 (2014). https://doi.org/10.3102/0013189X14545513

    Article  Google Scholar 

  20. Metcalf, S.J., Reilly, J.M., Kamarainen, A.M., King, J., Grotzer, T.A., Dede, C.: Supports for deeper learning of inquiry-based ecosystem science in virtual environments—comparing virtual and physical concept mapping. Comput. Hum. Behav. 87, 459–469 (2018). https://doi.org/10.1016/j.chb.2018.03.018

    Article  Google Scholar 

  21. Mislevy, R.J., Almond, R.G., Lukas, J.F.: A brief introduction to evidence-centered design. National Center for Research on Evaluation, Standards, and Student Testing (CRESST) Center for the Study of Evaluation (CSE), Graduate School of Education & Information Studies. (2004). https://files.eric.ed.gov/fulltext/ED483399.pdf

  22. National Research Council: Successful K-12 STEM Education: Identifying Effective Approaches in Science, Technology, Engineering, and Mathematics. The National Academies Press (2011). https://doi.org/10.17226/13158

  23. Noorden, R.V.: Researchers can get visibility and connections by putting their data online—if they go about it in the right way. Nature 500, 243–245 (2013). https://doi.org/10.1038/nj7461-243a

    Article  Google Scholar 

  24. Owen, V.E., Baker, R.S.: Fueling prediction of player decisions: foundations of feature engineering for optimized behavior modeling in serious games. Technol. Knowl. Learn. 25(2), 225–250 (2020). https://doi.org/10.1007/s10758-018-9393-9

    Article  Google Scholar 

  25. Owen, V.E., Ramirez, D., Salmon, A., Halverson, R.: Capturing learner trajectories in educational games through ADAGE (assessment data aggregator for game environments): a click-stream data framework for assessment of learning in play. In: American Educational Research Association Annual Meeting, Philadelphia (2014)

    Google Scholar 

  26. Owen, V.E., Roy, M.-H., Thai, K.P., Burnett, V., Jacobs, D., Baker, R.S.: Detecting wheel-spinning and productive persistence in educational games, p. 6 (2019)

    Google Scholar 

  27. Pfotenhauer, J.M., Gagnon, D.J., Litzkow, M.J., Blakesley, C.C.: Designing and using an on-line game to teach engineering. In: FIE2009, pp. 1–5 (2009)

    Google Scholar 

  28. Mayer, R.: Computer Games for Learning: An Evidence-Based Approach. MIT Press, Cambridge (2014)

    Book  Google Scholar 

  29. Rosenthal, R.: The file drawer problem and tolerance for null results. Psychol. Bull. 86(3), 638–641 (1979). https://doi.org/10.1037/0033-2909.86.3.638

    Article  Google Scholar 

  30. Rowe, E., et al.: Assessing implicit science learning in digital games. Comput. Hum. Behav. 76, 617–630 (2017). https://doi.org/10.1016/j.chb.2017.03.043

    Article  Google Scholar 

  31. Scianna, J., Gagnon, D., Knowles, B.: Counting the game: visualizing changes in play by incorporating game events. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 218–231. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_15

    Chapter  Google Scholar 

  32. Shaffer, D.W.: Epistemic frames for epistemic games. Comput. Educ. 46(3), 223–234 (2006). https://doi.org/10.1016/j.compedu.2005.11.003

    Article  Google Scholar 

  33. Shaffer, D.W., et al.: Epistemic network analysis: a prototype for 21st-century assessment of learning. Int. J. Learn. Media 1(2), 33–53 (2009). https://doi.org/10.1162/ijlm.2009.0013

    Article  MathSciNet  Google Scholar 

  34. Shute, V.J., Shute, V.J., Kim, Y.J.: Formative and stealth assessment. In: Spector, J., Merrill, M., Elen, J., Bishop, M. (eds.) Handbook of Research on Educational Communications and Technology, vol. 1, pp. 311–321. Springer, New York (2013). https://doi.org/10.1007/978-1-4614-3185-5_25

    Chapter  Google Scholar 

  35. Siemens, G., Baker, R.S.: Learning analytics and educational data mining: towards communication and collaboration. In: Proceedings of the 2nd International Conference on Learning Analytics and Knowledge - LAK 2012, p. 252 (2012). https://doi.org/10.1145/2330601.2330661

  36. Slater, S., Baker, R.S., Gagnon, D.J.: Changing students’ perceptions of a history exploration game using different scripts. In: Proceedings of the 30th International Conference on Computers in Education. International Conference on Computers in Education, Kuala Lumpur, Malaysia (2022)

    Google Scholar 

  37. Squire, K.D.: From content to context: videogames as designed experience. Educ. Res. 35(8), 19–29 (2006)

    Article  Google Scholar 

  38. Swanson, L., et al.: Leveraging cluster analysis to understand educational game player styles and support design. In: GLS 13.0 Conference Proceedings. Games, Learning, Society, Irving, CA (2022)

    Google Scholar 

  39. Swanson, L., Gagnon, D.J., Scianna, J.: A pilot study on teacher-facing real-time classroom game dashboards. In: International Conference on Meaningful Play, East Lansing, MI (2022)

    Google Scholar 

  40. Van Der Zee, T., Reich, J.: Open education science. AERA Open 4(3) (2018). https://doi.org/10.1177/2332858418787466

  41. Van ’T Veer, A.E., Giner-Sorolla, R.: Pre-registration in social psychology—a discussion and suggested template. J. Exp. Soc. Psychol. 67, 2–12. (2016). https://doi.org/10.1016/j.jesp.2016.03.004

  42. Ventura, M., Shute, V.: The validity of a game-based assessment of persistence. Comput. Hum. Behav. 29(6), 2568–2572 (2013). https://doi.org/10.1016/j.chb.2013.06.033

    Article  Google Scholar 

Download references

Acknowledgements

Open Game Data would not exist without the regular and thoughtful contributions of Erik Harpstead, Jennifer Scianna and Stefan Slater over the last years, as well as the many staff and interns who have worked on the project. We also acknowledge the insights brought by Ryan Baker, Matthew Berland, YJ Kim, Elizabeth Owen, and Dennis Ramirez at key moments that have guided the project.

This material is based upon work supported by the 2022 Learning Agency Tools Competition, the Wisconsin Department of Public Instruction and the National Science Foundation under Grant No. (1907384, 2116046, 2142103, 2243668).

Author information

Authors and Affiliations

  1. University of Wisconsin – Madison, Madison, WI, 53706, USA

    David J. Gagnon & Luke Swanson

Authors
  1. David J. Gagnon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luke Swanson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David J. Gagnon .

Editor information

Editors and Affiliations

  1. Trinity College Dublin, Dublin, Ireland

    Mads Haahr

  2. University of Costa Rica, San José, Costa Rica

    Alberto Rojas-Salazar

  3. TU Darmstadt, Darmstadt, Germany

    Stefan Göbel

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gagnon, D.J., Swanson, L. (2023). Open Game Data: A Technical Infrastructure for Open Science with Educational Games. In: Haahr, M., Rojas-Salazar, A., Göbel, S. (eds) Serious Games. JCSG 2023. Lecture Notes in Computer Science, vol 14309. Springer, Cham. https://doi.org/10.1007/978-3-031-44751-8_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-44751-8_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-44750-1

  • Online ISBN: 978-3-031-44751-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation