Sofia Strukova
Skip Abstract Section
Abstract
The rapid technological evolution of the last years has motivated students to develop capabilities that will prepare them for an unknown future in the 21st century. In this context, many teachers intend to optimise the learning process, making it more dynamic and exciting through the introduction of gamification. Thus, this article focuses on a data-driven assessment of geometry competencies, which are essential for developing problem-solving and higher-order thinking skills. Our main goal is to adapt, evaluate and compare Bayesian Knowledge Tracing (BKT), Performance Factor Analysis (PFA), Elo, and Deep Knowledge Tracing (DKT) algorithms applied to the data of a geometry game named Shadowspect, in order to predict students’ performance by means of several classifier metrics. We analysed two algorithmic configurations, with and without prioritisation of Knowledge Components (KCs) – the skills needed to complete a puzzle successfully, and we found Elo to be the algorithm with the best prediction power with the ability to model the real knowledge of students. However, the best results are achieved without KCs because it is a challenging task to differentiate between KCs effectively in game environments. Our results prove that the above-mentioned algorithms can be applied in formal education to improve teaching, learning, and organisational efficiency.
- [1] 2022. Common Core State Standards of Geometry. Retrieved 1 August 2022 from http://www.corestandards.org/Math/Content/G/Google Scholar
- [2] . 2018. Intelligent tutoring systems: A comprehensive historical survey with recent developments. arXiv:1812.09628. Retrieved from https://arxiv.org/abs/1812.09628Google Scholar
- [3] . 2020. Cheating behaviour in EFL classroom (a case study at elementary school in Sidenreng Rappang Regency). OKARA: Jurnal Bahasa dan Sastra 14, 2 (2020), 279–297.
DOI: Google ScholarCross Ref
- [4] . 2008. Why students engage in “gaming the system” behaviour in interactive learning environments. Journal of Interactive Learning Research 19, 2 (2008), 185–224.Google Scholar
- [5] . 2012. Towards automatically detecting whether student learning is shallow. In Proceedings of the International Conference on Intelligent Tutoring Systems. Springer, Springer, Berlin, 444–453. Google ScholarDigital Library
- [6] . 2016. Importance of gamification in increasing learning. The Journal of Continuing Education in Nursing 47, 8 (2016), 372–375.
DOI: Google ScholarCross Ref - [7] . 2018. Investigating the use of spatial reasoning strategies in geometric problem solving. International Journal of Technology and Design Education 29, 2(2018), 341–362.
DOI: Google ScholarCross Ref - [8] . 2020. Utilizing game analytics to inform and validate digital game-based assessment with evidence-centered game design: A case study. International Journal of Artificial Intelligence in Education 30, 3 (2020), 481–503.
DOI: Google ScholarCross Ref - [9] . 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and user-adapted Interaction 4, 4 (1994), 253–278.
DOI: Google ScholarCross Ref - [10] . 1997. Student modeling in the ACT programming tutor: Adjusting a procedural learning model with declarative knowledge. In Proceedings of the User Modeling. Springer, Springer Vienna, 243–254. Google ScholarCross Ref
- [11] . 2019. Analyzing student process data in game-based assessments with bayesian knowledge tracing and dynamic bayesian networks. Journal of Educational Data Mining 11, 1 (2019), 80–100.Google Scholar
- [12] . 2008. More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing. In Proceedings of the International Conference on Intelligent Tutoring Systems. Springer, Berlin, 406–415. Google ScholarDigital Library
- [13] . 2018. Combining Elo Rating and Collaborative Filtering to improve Learner Ability Estimation in an e-learning Context. Master’s thesis. NTNU.Google Scholar
- [14] . 2019. Intelligent tutoring system to improve learning outcomes. AI Communications 32, 3 (2019), 161–174.
DOI: Google ScholarDigital Library - [15] . 2018. Evaluation of the impact of deep learning architectural components selection and dataset size on a medical imaging task. In Proceedings of the Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. and (Eds.), Vol. 10579, International Society for Optics and Photonics, SPIE, 240–253.
DOI: Google ScholarCross Ref - [16] . 1978. The Rating of Chessplayers, Past and Present. Arco Pub.Google Scholar
- [17] . 2020. When is deep learning the best approach to knowledge tracing? JEDM| Journal of Educational Data Mining 12, 3 (2020), 31–54.
DOI: Google ScholarCross Ref - [18] . 2014. The relationship among geometry, working memory, and intelligence in children. Journal of Experimental Child Psychology 123 (2014), 112–128.
DOI: Google ScholarCross Ref - [19] . 2022. A systematic literature review of game-based assessment studies: Trends and challenges. IEEE Transactions on Learning Technologies (2022), 1–16.
DOI: Google ScholarCross Ref - [20] . 2013. Fast: Feature-aware student knowledge tracing. In Proceedings of the NIPS 2013 Workshop on Data Driven Education. University of Pittsburgh. Retrieved from http://d-scholarship.pitt.edu/20353/Google Scholar
- [21] . 2020. What explains the relationship between spatial and mathematical skills? A review of evidence from brain and behaviour. Psychonomic Bulletin Review 27, 3(2020), 465–482.
DOI: Google ScholarCross Ref - [22] . 2019. Game-based assessment: The past ten years and moving forward. In Game-Based Assessment Revisited. (2019), 3–11.
DOI: Google ScholarCross Ref - [23] . 2012. The knowledge-learning-instruction framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science 36, 5 (2012), 757–798.
DOI: Google ScholarCross Ref - [24] . 2009. Tensor decompositions and applications. SIAM Review 51, 3 (2009), 455–500.
DOI: Google ScholarDigital Library - [25] . 2021. Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning. Computers in Human Behaviour 125 (2021), 1–33.
DOI: Google ScholarDigital Library - [26] . 2022. Benefits of gamification in medical education. Clinical Anatomy 35, 6 (2022), 795–807.
DOI: Google ScholarCross Ref - [27] . 2017. Investigating the effect of an adaptive learning intervention on students’ learning. Educational Technology Research and Development 65, 6 (2017), 1605–1625.
DOI: Google ScholarCross Ref - [28] . 2021. A Survey of Knowledge Tracing. arXiv:2105.15106. Retrieved from https://arxiv.org/abs/2105.15106Google Scholar
- [29] . 2021. Tracing knowledge state with individual cognition and acquisition estimation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.Association for Computing Machinery, New York, NY, 173–182.
DOI: Google ScholarDigital Library - [30] . 2020. Visualizing educational game data: A case study of visualizations to support teachers. (June, 2020).
DOI: Google ScholarCross Ref - [31] . 2022. Interpretable knowledge tracing: Simple and efficient student modeling with causal relations. Proceedings of the AAAI Conference on Artificial Intelligence 36, 11(2022), 12810–12818.
DOI: Google ScholarCross Ref - [32] . 2018. Implementing flipped classroom that used an intelligent tutoring system into learning process. Computers and Education 124 (2018), 62–76.
DOI: Google ScholarCross Ref - [33] . 2019. Augmenting knowledge tracing by considering forgetting behaviour. In Proceedings of the World Wide Web Conference.Association for Computing Machinery, New York, NY, 3101–3107.
DOI: Google ScholarDigital Library - [34] . 2019. Elo rating algorithm for the purpose of measuring task difficulty in online learning environments. e-mentor5, 82 (2019), 43–51.
DOI: Google ScholarCross Ref - [35] . 2012. The sum is greater than the parts: Ensembling models of student knowledge in educational software. ACM SIGKDD Explorations Newsletter 13, 2 (2012), 37–44.
DOI: Google ScholarDigital Library - [36] . 2009. Performance factors analysis - A new alternative to knowledge tracing. Frontiers in Artificial Intelligence and Applications (2009), 531–538.
DOI: Google ScholarCross Ref - [37] . 2015. Metrics for evaluation of student models. Journal of Educational Data Mining 7, 2 (2015), 1–19.Google Scholar
- [38] . 2016. Applications of the Elo rating system in adaptive educational systems. Computers and Education 98 (2016), 169–179.
DOI: Google ScholarDigital Library - [39] . 2017. Bayesian knowledge tracing, logistic models, and beyond: An overview of learner modeling techniques. User Modeling and User-Adapted Interaction 27, 3 (2017), 313–350.
DOI: Google ScholarDigital Library - [40] . 2015. Deep knowledge tracing. Advances in Neural Information Processing Systems 28 (2015), 1–9. Retrieved from https://proceedings.neurips.cc/paper/2015/file/bac9162b47c56fc8a4d2a519803d51b3-Paper.pdfGoogle Scholar
- [41] . 2021. Ideating and developing a visualization dashboard to support teachers using educational games in the classroom. IEEE Access 9 (2021), 83467–83481.
DOI: Google ScholarCross Ref - [42] . 2020. Effects of solo vs. collaborative play in a digital learning game on geometry: Results from a K12 experiment. Computers and Education 159 (2020), 104008.
DOI: Google ScholarCross Ref - [43] . 2017. Detecting and clustering students by their gamification behaviour with badges: A case study in engineering education. International Journal of Engineering Education 33, 2-B (2017), 816–830.Google Scholar
- [44] . 2014. Parameterized exercises in java programming: Using knowledge structure for performance prediction. In Proceedings of the 2nd Workshop on AI-supported Education for Computer Science. University of Pittsburgh, 61–70.Google Scholar
- [45] . 2022. Empirical evaluation of deep learning models for knowledge tracing: Of hyperparameters and metrics on performance and replicability. Journal of Educational Data Mining 14, 2(2022), 32–102.
DOI: Google ScholarCross Ref - [46] . 2019. Extending Deep Knowledge Tracing: Inferring Interpretable Knowledge and Predicting Post-System Performance. arXiv:1910.12597. Retrieved from https://arxiv.org/abs/1910.12597Google Scholar
- [47] . 2021. Data-driven performance prediction in a geometry game environment. In Proceedings of the Conference on Information Technology for Social Good.Association for Computing Machinery, New York, NY, 283–288.
DOI: Google ScholarDigital Library - [48] . 2022. A survey on data-driven evaluation of competencies and capabilities across multimedia environments. International Journal of Interactive Multimedia and Artificial Intelligence (2022). Google ScholarCross Ref
- [49] . 2019. On-the-fly calibration in computerized adaptive testing. In Proceedings of the Theoretical and Practical Advances in Computer-based Educational Measurement. Springer International Publishing, 307–323.
DOI: Google ScholarCross Ref - [50] . 2012. Flood risk and social justice. IWA Publishing.Google Scholar
- [51] . 2019. Deep knowledge tracing with side information. In Proceedings of the International Conference on Artificial Intelligence in Education. Springer, Springer International Publishing, 303–308. Google ScholarDigital Library
- [52] . 2010. Monitoring learners’ proficiency: Weight adaptation in the Elo rating system. In Proceedings of the Educational Data Mining 2011.Google Scholar
- [53] . 1991. Motivation and academic achievement: The effects of personality traits and the duality of experience. Journal of Personality 59, 3 (1991), 539–574.
DOI: Google ScholarCross Ref - [54] . 2023. A bayesian-based knowledge tracing model for improving safety training outcomes in construction: An adaptive learning framework. Developments in the Built Environment 13 (2023), 100111.
DOI: Google ScholarCross Ref - [55] . 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proceedings of the 5th Annual ACM Conference on Learning at Scale.Association for Computing Machinery, New York, NY, 10 pages.
DOI: Google ScholarDigital Library - [56] . 2017. Incorporating rich features into deep knowledge tracing. In Proceedings of the 4th (2017) ACM Conference on Learning @ Scale.Association for Computing Machinery, New York, NY, 169–172.
DOI: Google ScholarDigital Library
Index Terms
- Adapting Knowledge Inference Algorithms to Measure Geometry Competencies through a Puzzle Game
Recommendations
Using serious games to manage knowledge and competencies: The seven-step development process
This article explores how serious games improve knowledge and competencies management in the context of human resources management. The exploratory research, based on the conceptual framework of the SECI model from Nonaka, analyzes the performances of ...
Essential competencies for computing managers: Skills and dispositions
AbstractCompetencies (knowledge, skills, and dispositions) enable employers and educators to speak a common language regarding what computing graduates are expected to demonstrate on the job. This study focuses on competencies required by managers in the ...
Differentiating between Leadership Competencies and Styles: A Critical Review in Project Management Perspective
Literature reveals a number of confusions within the area of project management regarding the use of terminology and differing interpretations, concerning leadership competencies and leadership styles. In project management literature, many empirical ...
Comments