Skip to main content
SpringerLink
Log in

Cracking the Code of Learning Gains: Using Ordered Network Analysis to Understand the Influence of Prior Knowledge

  • Conference paper
  • First Online:
Advances in Quantitative Ethnography (ICQE 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1895))

Included in the following conference series:

Abstract

Prior research has shown that digital games can enhance STEM education by providing learners with immersive and authentic scientific experiences. However, optimizing the learning outcomes of students engaged in game-based environments requires aligning the game design with diverse student needs. Therefore, an in-depth understanding of player behavior is crucial for identifying students who need additional support or modifications to the game design. This study applies an Ordered Network Analysis (ONA)—a specific kind of Epistemic Network Analysis (ENA)—to examine the game trace log data of student interactions, to gain insights into how learning gains relate to the different ways that students move through an open-ended virtual world for learning microbiology. Our findings reveal that differences between students with high and low learning gains are mediated by their prior knowledge. Specifically, level of prior knowledge is related to behaviors that resemble wheel-spinning, which warrant the development of future interventions. Results also have implications for discovery with modeling approaches and for enhancing in-game support for learners and improving game design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Clark, D.B., Sengupta, P., Brady, C.E., Martinez-Garza, M.M., Killingsworth, S.S.: Disciplinary integration of digital games for science learning. Int. J. STEM Educ. 2(1), 2 (2015)

    Article  Google Scholar 

  2. Gao, F., Li, L., Sun, Y.: A systematic review of mobile game-based learning in STEM education. Educ. Tech. Res. Dev. 68, 1791–1827 (2020)

    Article  Google Scholar 

  3. Saricam, U., Yildirim, M.: The effects of digital game-based STEM activities on students’ interests in STEM fields and scientific creativity: minecraft case. Int. J. Technol. Educ. Sci. 5(2), 166–192 (2021)

    Article  Google Scholar 

  4. Shaffer, D.W., Gee, J.P.: How Computer Games help Children Learn. Palgrave Macmillan, New York (2006)

    Book  Google Scholar 

  5. Annetta, L.A., Minogue, J., Holmes, S.Y., Cheng, M.T.: Investigating the impact of video games on high school students’ engagement and learning about genetics. Comput. Educ. 53(1), 74–85 (2009)

    Article  Google Scholar 

  6. Anderson, J., Barnett, M.: Using video games to support pre-service elementary teachers learning of basic physics principles. J. Sci. Educ. Technol. 20, 347–362 (2011)

    Article  Google Scholar 

  7. Kanematsu, H., Kobayashi, T., Barry, D.M., Fukumura, Y., Dharmawansa, A., Ogawa, N.: Virtual STEM class for nuclear safety education in metaverse. Procedia Comput. Sci. 35, 1255–1261 (2014)

    Article  Google Scholar 

  8. Owen, V.E., et al.: Detecting wheel spinning and productive persistence in educational games. In: Proceedings of the 12th International Conference on Educational Data Mining, pp. 378–383 (2019)

    Google Scholar 

  9. Aleven, V., Stahl, E., Schworm, S., Fischer, F., Wallace, R.: Help seeking and help design in interactive learning environments. Rev. Educ. Res. 73(3), 277–320 (2003)

    Article  Google Scholar 

  10. Nietfeld, J.L., Shores, L.R., Hoffmann, K.F.: Self-regulation and gender within a game-based learning environment. J. Educ. Psychol. 106(4), 961 (2014)

    Article  Google Scholar 

  11. Zhang, J., et al.: Investigating student interest and engagement in game-based learning environments. In: Artificial Intelligence in Education: 23rd International Conference, AIED 2022, Durham, UK, July 27–31, 2022, Proceedings, Part I, pp. 711–716. Springer International Publishing (2022)

    Google Scholar 

  12. Steinkuehler, C.: Video games and digital literacies. J. Adolesc. Health. 54(1), 61–63 (2010)

    Google Scholar 

  13. Sawyer, R., Rowe, J., Azevedo, R., Lester, J.: Filtered Time Series Analyses of Student Problem-Solving Behaviors in Game-Based Learning. International Educational Data Mining Society (2018)

    Google Scholar 

  14. Nasiar, N. et al.: It’s good to explore: investigating silver pathways and the role of frustration during game-based learning. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds.) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2023. Communications in Computer and Information Science, vol. 1831. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-36336-8_77

  15. Shaffer, D.W., et al.: Epistemic network analysis: a prototype for 21st-century assessment of learning. Int. J. Learn. Media 1(2), 33–53 (2009)

    Article  MathSciNet  Google Scholar 

  16. Rowe, J.P., Shores, L.R., Mott, B.W., Lester, J.C.: Integrating learning, problem solving, and engagement in narrative-centered learning environments. Int. J. Artif. Intell. Educ. 21(1–2), 115–133 (2011)

    Google Scholar 

  17. Reilly, J.M., Dede, C.: Differences in student trajectories via filtered time series analysis in an immersive virtual world. In: Proceedings of the 9th International Conference on Learning Analytics and Knowledge, pp. 130–134 (2019)

    Google Scholar 

  18. Abdi, H., Williams, L.J.: Principal component analysis. WIRs: Comput. Stat. 2(4), 433–459 (2010)

    Google Scholar 

  19. Shaffer, D.W., Collier, W., Ruis, A.R.: A tutorial on epistemic network analysis: analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Analytics 3(3), 9–45 (2016)

    Article  Google Scholar 

  20. Barany, A., Foster, A.: Examining identity exploration in a video game participatory culture. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) ICQE 2019. CCIS, vol. 1112, pp. 3–13. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33232-7_1

    Chapter  Google Scholar 

  21. Bressler, D.M., Bodzin, A.M., Eagan, B., Tabatabai, S.: Using epistemic network analysis to examine discourse and scientific practice during a collaborative game. J. Sci. Educ. Technol. 28, 553–566 (2019)

    Article  Google Scholar 

  22. Bressler, D.M., Annetta, L.A., Dunekack, A., Lamb, R.L., Vallett, D.B.: How STEM game design participants discuss their project goals and their success differently. In: Advances in Quantitative Ethnography: 3rd International Conf., ICQE 2021, Virtual Event, November 6–11, 2021, Proceedings 3, pp. 176–190. Springer International Publishing (2022). https://doi.org/10.1007/978-3-030-93859-8_12

  23. Karumbaiah, S., Baker, R.S., Barany, A., Shute, V.: Using epistemic networks with automated codes to understand why players quit levels in a learning game. In: Eagan, B., Misfeldt, M., Siebert-Evenstone, A. (eds.) Advances in Quantitative Ethnography. ICQE 2019. Communications in Computer and Information Science, vol. 1112. Springer, Cham (2019).https://doi.org/10.1007/978-3-030-33232-7_9

  24. Scianna, J., Gagnon, D., Knowles, B.: Counting the game: visualizing changes in play by incorporating game events. In: Ruis, A.R., Lee, S.B. (eds.) Advances in Quantitative Ethnography. ICQE 2021. Communications in Computer and Information Science, vol. 1312. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_15

  25. Tan, Y., Ruis, A.R., Marquart, C., Cai, Z., Knowles, M., Shaffer, D.W: Ordered network analysis. In: Damşa, C., Barany, A. (eds.) Advances in Quantitative Ethnography: Fourth International Conference, ICQE 2022. Springer International Publishing (2022). https://doi.org/10.1007/978-3-031-31726-2_8

  26. Wan, H., Beck, J.B.: Considering the Influence of Prerequisite Performance on Wheel Spinning. International Educational Data Mining Society (2015)

    Google Scholar 

  27. Palaoag, T.D., Rodrigo, M.M.T., Andres, J.M.L., Andres, J.M.A.L., Beck, J.E.: Wheel-spinning in a game-based learning environment for physics. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 234–239. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39583-8_23

    Chapter  Google Scholar 

  28. Fan, Y., et al.: Dissecting learning tactics in MOOC using ordered network analysis. J. Comput. Assist. Learn. 39(1), 154–166 (2023)

    Article  Google Scholar 

  29. Min, W., et al.: Multimodal goal recognition in open-world digital games. Proc. AAAI Conf. Artif. Intell. Interact. Dig. Entertainment 13(1), 80–86 (2021)

    Google Scholar 

  30. Vail, A.K., Grafsgaard, J.F., Boyer, K.E., Wiebe, E.N., Lester, J.C.: Predicting Learning from student affective response to tutor questions. In: Micarelli, A., Stamper, J., Panourgia, K. (eds.) ITS 2016. LNCS, vol. 9684, pp. 154–164. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39583-8_15

    Chapter  Google Scholar 

  31. Wang, Y., Swiecki, Z., Ruis, A.R., Shaffer, D.W.: Simplification of epistemic networks using parsimonious removal with interpretive alignment. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 137–151. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_10

    Chapter  Google Scholar 

  32. Arastoopour Irgens, G., Shaffer, D.W., Swiecki, Z., Ruis, A.R., Chesler, N.C.: Teaching and assessing engineering design thinking with virtual internships and epistemic network analysis. Int. J. Eng. Educ. 32, 1492–1501 (2015)

    Google Scholar 

  33. Beck, J.E., Gong, Y.: Wheel-spinning: Students who fail to master a skill. In: Artificial Intelligence in Education: 16th International Conference AIED 2013, Memphis, TN, USA, July 9–13, 2013, pp. 431–440. Springer, Berlin Heidelberg (2013)

    Chapter  Google Scholar 

  34. Yang, J.C., Quadir, B.: Effects of prior knowledge on learning performance and anxiety in an English learning online role-playing game. J. Educ. Technol. Soc. 21(3), 174–185 (2018)

    Google Scholar 

  35. Kai, S., Almeda, M.V., Baker, R.S., Heffernan, C., Heffernan, N.: Decision tree modeling of wheel-spinning and productive persistence in skill builders. J. Educ. Data Min. 10(1), 36–71 (2018)

    Google Scholar 

  36. Botelho, A.F., Varatharaj, A., Patikorn, T., Doherty, D., Adjei, S.A., Beck, J.E.: Developing early detectors of student attrition and wheel spinning using deep learning. IEEE Trans. Learn. Technol. 12(2), 158–170 (2019)

    Article  Google Scholar 

  37. Baker, R.S., Yacef, K.: The state of educational data mining in 2009: a review and future visions. J. Educ. Data Min. 1(1), 3–17 (2009)

    Google Scholar 

  38. Brohinsky, J., Marquart, C., Wang, J., Ruis, A.R., Shaffer, D.W.: Trajectories in epistemic network analysis. In: Ruis, A.R., Lee, S.B. (eds.) ICQE 2021. CCIS, vol. 1312, pp. 106–121. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67788-6_8

    Chapter  Google Scholar 

  39. Squire, K.D.: Video-game literacy: A literacy of expertise. In: Handbook of Research on New Literacies, pp. 635–669. Routledge (2014)

    Google Scholar 

  40. Kress, S., Neudorf, J., Borowsky, B., Borowsky, R.: What’s in a game: video game visual-spatial demand location exhibits a double dissociation with reading speed. Acta Physiol. (Oxf) 232, 103822 (2023)

    Google Scholar 

Download references

Acknowledgments

This work was funded by a grant from the National Science Foundation (NSF Cyberlearning #2016943). We would also like to thank Kristy Boyer for access to this data and Yuanru Tan for developing the ONA package used in this analysis. Andres Felipe Zambrano thanks the Ministerio de Ciencia, Tecnología e Innovación and the Fulbright-Colombia commission for supporting his doctoral studies through the Fulbright-MinCiencias 2022 scholarship.

Author information

Authors and Affiliations

  1. University of Pennsylvania, Philadelphia, PA, USA

    Andres Felipe Zambrano, Amanda Barany, Jaclyn Ocumpaugh & Nidhi Nasiar

  2. University of Denver, Denver, CO, USA

    Stephen Hutt

  3. North Carolina State University, Raleigh, NC, USA

    Alex Goslen, Jonathan Rowe, James Lester, Eric Wiebe & Bradford Mott

Authors
  1. Andres Felipe Zambrano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amanda Barany
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jaclyn Ocumpaugh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nidhi Nasiar
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephen Hutt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alex Goslen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jonathan Rowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. James Lester
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Eric Wiebe
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Bradford Mott
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andres Felipe Zambrano .

Editor information

Editors and Affiliations

  1. Clemson University, Clemson, SC, USA

    Golnaz Arastoopour Irgens

  2. University of Technology Sydney, Ultimo, NSW, Australia

    Simon Knight

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zambrano, A.F. et al. (2023). Cracking the Code of Learning Gains: Using Ordered Network Analysis to Understand the Influence of Prior Knowledge. In: Arastoopour Irgens, G., Knight, S. (eds) Advances in Quantitative Ethnography. ICQE 2023. Communications in Computer and Information Science, vol 1895. Springer, Cham. https://doi.org/10.1007/978-3-031-47014-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47014-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47013-4

  • Online ISBN: 978-3-031-47014-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation