Learning Analytics and Educational Games: Lessons Learned from Practical Experience

Abstract

Learning Analytics (LA) is an emerging discipline focused on obtaining information by analyzing students’ interactions with on-line educational contents. Data is usually collected from online activities such as forums or virtualized courses hosted on Learning Management Systems (e.g. Moodle). Educational games are emerging as a popular type of e-learning content and their high interactivity makes them potential sources for relevant educational user data. However, it is still uncertain how to deploy and combine these two incipient technologies, as multiple challenges remain unresolved. This paper reports on our practical experience using LA to improve assessment of experimental research on educational game-based instruction. In the last year, we conducted four experiments evaluating game-based instruction under different conditions (using three adventure games and a puzzle game respectively) in 13 educational institutions including schools, universities and vocational training organizations. A LA system was used to track interaction around six hundred students. In these experiences, we encountered several problems in each of the steps of the process, from issues related to the design of the experiment and the game to different technical and practical problems, derived from the very diverse conditions of the facilities and policies of each institution (computer laboratories, computers hardware, software installed, irregular Internet access), which hinders the data collecting process (e.g. the system had to deal with high latency Internet connections and backup plans had to be devised for collecting data when no Internet access was available). We present the lessons learned and propose guidelines from a technical and practical perspective for the design of experimental research using both LA systems and educational games.