Skip to main content
SpringerLink
Log in

Exploring Design Concepts to Enable Teachers to Monitor and Adapt Gamification in Adaptive Learning Systems: A Qualitative Research Approach

  • ARTICLE
  • Published:
International Journal of Artificial Intelligence in Education Aims and scope Submit manuscript

Abstract

There is growing interest in applying gamification to adaptive learning systems to motivate and engage students during the learning process. However, previous studies have reported unexpected results about student outcomes in these systems. One of the causes of these unfavorable effects is the lack of monitoring and adaptation of gamification design when students do not achieve the expected objectives during the learning process. Based on this, this paper explores twenty design concepts to enable teachers to monitor and adapt the gamification design of adaptive learning systems. This research uses the speed dating method with fifteen teachers to validate these concepts in a consistent way with the “gamification analytics model for teachers.” According to this model, teachers can define interaction goals, monitor students’ interaction with learning resources and gamification elements, and adapt the gamification design by creating missions to help students achieve defined interaction goals. The results of this research indicate that teachers found it valuable and relevant to visualize students’ interaction with gamification elements, such as missions and levels, to help them understand their students’ status. In contrast, they poorly evaluated the visualization of students’ interaction with the trophies. Teachers also highly considered creating customized missions for a student or a specific group to help students engage and achieve the desired learning goals. The teachers’ opinions on the design concepts provide relevant insights to support them in the monitoring and adaptation phases of gamification in adaptive educational systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://avance.eyeduc.com/

References

  • Baker, R.S., D’Mello, S.K., Rodrigo, M.T., & Graesser, A.C. (2010). Better to be frustrated than bored: The incidence, persistence, and impact of learners‘ cognitive - affective states during interactions with three different computer-based learning environments. International Journal of Human-Computer Studies, 68 (4), 223–241. https://doi.org/10.1016/j.ijhcs.2009.12.003. http://www.sciencedirect.com/science/article/pii/S1071581909001797.

    Article  Google Scholar 

  • Bell, C., & Mcnamara, D. (2007). Integrating iSTART into a high school curriculum. In Proceedings of the 29th annual meeting of the cognitive science society, Austin, TX, pp. 809–814.

  • Borges, S.S., Mizoguchi, R., Durelli, V.H.S., Bittencourt, I.I., & Isotani, S. (2016). A link between worlds: Towards a conceptual framework for bridging player and learner roles in gamified collaborative learning contexts. In F Koch, A Koster, T Primo, & C Guttmann (Eds.) Advances in social computing and digital education (pp. 19–34). Cham: Springer International Publishing.

  • Borras-Gene, O., Martinez-Nunez, M., & Blanco, A. (2016). New challenges for the motivation and learning in engineering education using gamification in MOOC. International Journal of Engineering Education, 32, 501–512.

    Google Scholar 

  • du Boulay, B. (2016). Recent Meta-reviews and Meta–analyses of AIED Systems. International Journal of Artificial Intelligence in Education, 26(1), 536–537. https://doi.org/10.1007/s40593-015-0060-1.

    Article  Google Scholar 

  • Brusilovsky, P., & Peylo, C. (2003). Adaptive and intelligent web-based educational systems. International Journal of Artificial Intelligence in Education, 13(2-4), 159–172. http://dl.acm.org/citation.cfm?id=1434845.1434847.

    Google Scholar 

  • Buchenau, M., & Suri, J.F. (2000). Experience prototyping. In Proceedings of the conference on designing interactive systems processes practices methods and techniques DIS, 00, October, 424–433. https://doi.org/10.1145/347642.347802.

  • Calderón, A, Boubeta-Puig, J., & Ruiz, M. (2018). MEDit4CEP-gam: A model-driven approach for user-friendly gamification design, monitoring and code generation in CEP-based systems. Information and Software Technology, 95, 238–264. https://doi.org/10.1016/j.infsof.2017.11.009.

    Article  Google Scholar 

  • Cassano, F., Piccinno, A., Roselli, T., & Rossano, V. (2019). Gamification and learning analytics to improve engagement in university courses. In T. Di Mascio, P. Vittorini, R. Gennari, F. De la Prieta, S. Rodríguez, M. Temperini, R. Azambuja Silveira, E. Popescu, & L. Lancia (Eds.) Methodologies and intelligent systems for technology enhanced learning, 8th international conference (pp. 156–163). Cham: Springer International Publishing.

  • Chalco, G., Moreira, D.A., Mizoguchi, R., & Isotani, S. (2014). An ontology engineering approach to gamify collaborative learning scenarios. In Lecture notes in computer science, pp. 193–206.

  • Codish, D., & Ravid, G. (2014). Adaptive approach for gamification optimization. In 2014 IEEE/ACM 7th international conference on utility and cloud computing, pp. 609–610 https://doi.org/10.1109/UCC.2014.94.

  • Cukurova, M., Kent, C., & Luckin, R. (2019). Artificial intelligence and multimodal data in the service of human decision - making: a case study in debate tutoring. British Journal of Educational Technology, 50(6), 3032–3046. https://doi.org/10.1111/bjet.12829.

    Article  Google Scholar 

  • Davidoff, S., Lee, M.K., Dey, A.K., & Zimmerman, J. (2007). Rapidly exploring application design through speed dating. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 4717 LNCS, pp. 429–446 https://doi.org/10.1007/978-3-540-74853-3_25.

  • Dermeval, D., & Bittencourt, I.I. (2020). Co-designing gamified intelligent tutoring systems with teachers. Revista Brasileira de Informá,tica na Educação, 28, 73.

    Article  Google Scholar 

  • Dermeval, D., Paiva, R., Bittencourt, I.I., Vassileva, J., & Borges, D. (2018). Authoring tools for designing intelligent tutoring systems: a systematic review of the literature. International Journal of Artificial Intelligence in Education, 28(3), 336–384.

    Article  Google Scholar 

  • Dermeval, D., Albuquerque, J., Bittencourt, I.I., Isotani, S., Silva, A.P., & Vassileva, J. (2019). Gato: An ontological model to apply gamification in intelligent tutoring systems. Frontiers in Artificial Intelligence, 2, 13. https://doi.org/10.3389/frai.2019.00013. https://www.frontiersin.org/article/10.3389/frai.2019.00013.

    Article  Google Scholar 

  • Dillahunt, T.R., Lam, J., Lu, A., & Wheeler, E. (2018). Designing Future employment applications for underserved job seekers: A speed dating study. In DIS 2018 - Proceedings of the 2018 designing interactive systems conference, association for computing machinery, Inc, pp. 33–44. https://doi.org/10.1145/3196709.3196770.

  • Fang, Y., Ren, Z., Hu, X., & Graesser, A.C. (2018). A meta-analysis of the effectiveness of aleks on learning. Educational Psychology pp No Pagination Specified–No Pagination Specified. https://doi.org/10.1080/01443410.2018.1495829.

  • Ferro, L.S., Walz, S.P., & Greuter, S. (2013). Towards personalised, gamified systems: An investigation into game design, personality and player typologies. In Proceedings of The 9th australasian conference on interactive entertainment: matters of life and death, ACM, New York, NY, USA, IE ’13, p 7:1–7:6 https://doi.org/10.1145/2513002.2513024.

  • González, C., Toledo, P., & Muñoz, V. (2016). Enhancing the engagement of intelligent tutorial systems through personalization of gamification. International Journal of Engineering Education, 32(1), 532–541.

    Google Scholar 

  • Hallifax, S., Serna, A., Marty, J.C., & Lavoue, E. (2019). Adaptive gamification in education: a literature review of current trends and developments. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-030-29736-7_22, (Vol. 11722 LNCS pp. 294–307). Springer Verlag.

  • Heilbrunn, B., Herzig, P., & Schill, A. (2014a). Tools for Gamification Analytics: A Survey. In 2014 IEEE/ACM 7th international conference on utility and cloud computing, pp. 603–608 https://doi.org/10.1109/UCC.2014.93.

  • Heilbrunn, B., Herzig, P., & Schill, A. (2014b). Towards gamification analytics-requirements for monitoring and adapting gamification designs. GI-Jahrestagung.

  • Heilbrunn, B., Herzig, P., & Schill, A. (2017). Gamification analytics - methods and tools for monitoring and adapting gamification designs. In S. Stieglitz, C. Lattemann, S. Robra-Bissantz, R. Zarnekow, & T. Brockmann (Eds.) Gamification: Using game elements in serious contexts. https://doi.org/10.1007/978-3-319-45557-0_3 (pp. 31–47). Cham.

  • Holstein, K., McLaren, B.M., & Aleven, V. (2017). Intelligent tutors as teachers’ aides: Exploring teacher needs for real-time analytics in blended classrooms. In ACM International conference proceeding series, association for computing machinery, pp. 257–266 https://doi.org/10.1145/3027385.3027451.

  • Holstein, K., McLaren, B.M., & Aleven, V. (2019). Designing for complementarity: Teacher and student needs for orchestration support in AI-enhanced classrooms. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics). https://doi.org/10.1007/978-3-030-23204-7_14, (Vol. vol 11625 LNAI pp. 157–171). Springer Verlag.

  • Huang, B., & Hew, K.F. (2018). Implementing a theory-driven gamification model in higher education flipped courses: Effects on out-of-class activity completion and quality of artifacts. Computers & Education, 125, 254–272. https://doi.org/10.1016/j.compedu.2018.06.018.

    Article  Google Scholar 

  • Izumi, L., Fathers, F., & Clemens, J. (2013). Technology and Education: A primer, Fraser Institute, Canada.

  • Klock, A.C.T., Gasparini, I., Pimenta, M.S., & Hamari, J. (2020). Tailored gamification: a review of literature. International Journal of Human Computer Studies, 144, 102495. https://doi.org/10.1016/j.ijhcs.2020.102495.

    Article  Google Scholar 

  • Kulik, J.A., & Fletcher, J.D. (2016). Effectiveness of intelligent tutoring systems: a Meta-Analytic review. Review of Educational Research, 86(1), 42–78. https://doi.org/10.3102/0034654315581420.

    Article  Google Scholar 

  • Kurcz, J.M., Chang, T., & Graf, S. (2015). Improving communication and project management through an adaptive collaborative learning system. In 2015 IEEE 15th International conference on advanced learning technologies, pp. 61–63 https://doi.org/10.1109/ICALT.2015.140.

  • Latulipe, C., Long, N.B., & Seminario, C.E. (2015). Structuring flipped classes with lightweight teams and gamification. In Proceedings of the 46th ACM Technical symposium on computer science education, ACM, New York, NY, USA, SIGCSE ’15, pp. 392–397 https://doi.org/10.1145/2676723.2677240.

  • Lavoue, E., Monterrat, B., Desmarais, M., & George, S. (2018). Adaptive gamification for learning environments. IEEE Transactions on Learning Technologies 1. https://doi.org/10.1109/TLT.2018.2823710.

  • van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL: Balancing between overview and overload. Journal of Learning Analytics, 2, 138–162. https://doi.org/10.18608/jla.2015.22.11.

    Article  Google Scholar 

  • Ma, W., Adesope, O.O., Nesbit, J.C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: a meta-analysis. Journal of Educational Psychology, 106(4), 901–918. https://doi.org/10.1037/a0037123.

    Article  Google Scholar 

  • Macleod, H., & Sinclair, C. (2017). Digital learning and the changing role of the teacher. In Encyclopedia of educational philosophy and theory. https://doi.org/10.1007/978-981-287-588-4_126 (pp. 566–571). Singapore: Springer Singapore.

  • Magnisalis, I., Demetriadis, S., & Karakostas, A. (2011). Adaptive and intelligent systems for collaborative learning support: a review of the field. IEEE Transactions on Learning Technologies, 4(1), 5–20. https://doi.org/10.1109/TLT.2011.2.

    Article  Google Scholar 

  • Molenaar, I., & Knoop-van Campen, C. (2017). Teacher dashboards in practice: Usage and impact. In E. Lavoue, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.) Data driven approaches in digital education (pp. 125–138). Cham: Springer International Publishing.

  • Molenaar, I., & Knoop-van Campen, C. (2018). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies 1–1. https://doi.org/10.1109/TLT.2018.2851585.

  • Monterrat, B., Lavoue, E., & George, S. (2017). Adaptation of gaming features for motivating learners. Simulation & Gaming, 48(5), 625–656. https://doi.org/10.1177/1046878117712632.

    Article  Google Scholar 

  • Mora, A., Riera, D., González, C., & Arnedo-Moreno, J. (2017). Gamification: a systematic review of design frameworks. Journal of Computing in Higher Education, 29(3), 516–548. https://doi.org/10.1007/s12528-017-9150-4.

    Article  Google Scholar 

  • Muñoz, G, Merchan, L., Jurado, J., & Collazos, C. (2016). Estimating the use of gamification in collaborative moocs, a methodological proposal. In 8Th international conference on education and new learning technologies, IATED, Barcelona, Spain, pp. 8703–8712.

  • Nacke, L.E., Bateman, C., & Mandryk, R.L. (2014). Brainhex: A neurobiological gamer typology survey. Entertainment Computing, 5(1), 55–62. https://doi.org/10.1016/j.entcom.2013.06.002.

    Article  Google Scholar 

  • Naik, V., & Kamat, V. (2015). Adaptive and gamified learning environment (AGLE). In 2015 IEEE Seventh international conference on technology for education (T4E), pp. 7–14 https://doi.org/10.1109/T4E.2015.23.

  • Odom, W., Zimmerman, J., Davidoff, S., Forlizzi, J., Dey, A.K., & Lee, M.K. (2012). A fieldwork of the future with user enactments. In Proceedings of the designing interactive systems conference, DIS ’12, pp. 338–347 https://doi.org/10.1145/2317956.2318008.

  • Paiva, J., Leal, J., & Queirós, R (2016a). Enki: A Pedagogical Services Aggregator For Learning Programming Languages,. In 21St annual conference on innovation and technology in computer science education (ITiCSE2016), ACM. Arequipa, Peru, pp. 332–337.

  • Paiva, R., & Bittencourt, I.I. (2017). The authoring of pedagogical decisions informed by data, on the perspective of a MOOC. In Anais dos workshops do VI congresso brasileiro de informática na educação (CBIE 2017), brazilian computer society (sociedade brasileira de computação - SBC). https://doi.org/10.5753/cbie.wcbie.2017.15, (Vol. 1 p. 15).

  • Paiva, R., & Bittencourt, I.I. (2020). Helping teachers help their students: A human-ai hybrid approach. In International conference on artificial intelligence in education. https://doi.org/10.1007/978-3-030-52237-7_36 (pp. 448–459). Springer.

  • Paiva, R., Bittencourt, I.I., Tenrio, T., Jaques, P., & Isotani, S. (2016b). What do students do on-line? modeling students’ interactions to improve their learning experience. Computers in Human Behavior, 64(C), 769–781. https://doi.org/10.1016/j.chb.2016.07.048.

    Article  Google Scholar 

  • Paiva, R.O.A., Bittencourt, I.I., da, Silva AP, Isotani, S., & Jaques, P. (2015). Improving pedagogical recommendations by classifying students according to their interactional behavior in a gamified learning environment. In Proceedings of the 30th annual ACM symposium on applied computing, ACM, New York, NY, USA, SAC ’15, pp. 233–238 https://doi.org/10.1145/2695664.2695874.

  • Prenger, R., & Schildkamp, K. (2018). Data-based decision making for teacher and student learning: a psychological perspective on the role of the teacher. Educational Psychology, 38(6), 734–752. https://doi.org/10.1080/01443410.2018.1426834.

    Article  Google Scholar 

  • Santos, W., Bittencourt, I.I., & Vassileva, J. (2018). Design of tailored gamified educational systems based on gamer types. In Anais dos workshops do congresso brasileiro de informática na educação. https://doi.org/10.5753/cbie.wcbie.2018.42, (Vol. 7 p. 42).

  • Shi, L., & Cristea, A.I. In A Micarelli, J Stamper, & K Panourgia (Eds.) (2016). Motivational gamification strategies rooted in Self-Determination theory for social adaptive E-Learning. Cham: Springer International Publishing.

  • Snow, E.L., Allen, L.K., Jackson, G.T., & McNamara, D.S. (2015). Spendency: Students‘ propensity to use system currency. International Journal of Artificial Intelligence in Education, 25(3), 407–427. https://doi.org/10.1007/s40593-015-0044-1.

    Article  Google Scholar 

  • Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on k–12 students’ mathematical learning. Journal of Educational Psychology, 105(4), 970–987. https://doi.org/10.1037/a0032447.

    Article  Google Scholar 

  • Steenbergen-Hu, S., & Cooper, H. (2014). A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning. Journal of Educational Psychology, 106(2), 331–347. https://doi.org/10.1037/a0034752.

    Article  Google Scholar 

  • Tenório, K., Challco, C.G., Dermeval, D., Lemos, B., Nascimento, P., Santos, R., & da Silva, P.A. (2020). Helping teachers assist their students in gamified adaptive educational systems: Towards a gamification analytics tool. In Artificial intelligence in education. https://doi.org/10.1007/978-3-030-52240-7_57(pp. 312–317). Cham: Springer.

  • Tenório, K., Dermeval, D., Challco, G., Lemos, B., Nascimento, P., Santos, R., & Silva, A. (2020). An evaluation of the gamanalytics tool: is the gamification analytics model ready for teachers?. In Anais do XXXI Simpósio brasileiro de informática na educação, SBC, Porto Alegre, RS, Brasil, pp. 562–571. https://doi.org/10.5753/cbie.sbie.2020.562.

  • Tenório, T, Bittencourt, I.I., Isotani, S., Pedro, A., & Ospina, P. (2016). A gamified peer assessment model for on-line learning environments in a competitive context. Computers in Human Behavior, 64(C), 247–263. https://doi.org/10.1016/j.chb.2016.06.049.

    Article  Google Scholar 

  • Trinidad, M., Calderón, A., & Ruiz, M. (2018). A systematic literature review on the gamification monitoring phase: how spi standards can contribute to gamification maturity. In Software process improvement and capability determination, pp. 31–44 https://doi.org/10.1007/978-3-030-00623-5_3.

  • Truong, K.N., Hayes, G.R., & Abowd, G.D. (2006). Storyboarding: an empirical determination of best practices and effective guidelines. In Proceedings of DIS, association for computing machinery (ACM), p 12 https://doi.org/10.1145/1142405.1142410.

  • UNICEF. (2018). Raising Learning Outcomes: the opportunities and challenges of ICT for learning. Tech. rep.. https://www.unicef.org/esa/media/2636/file/UNICEF-AKF-IU-2018-ICT-Education-WCAR-ESAR.pdf.

  • Utomo, A.Y., & Santoso, H.B. (2015). Development of Gamification-enriched Pedagogical Agent for e-Learning System Based on Community of Inquiry. In Proceedings of the international HCI and UX conference in indonesia, ACM, New York, NY, USA, CHIuXiD ’15, pp. 1–9 https://doi.org/10.1145/2742032.2742033.

  • Vandewaetere, M., Desmet, P., & Clarebout, G. (2011). The contribution of learner characteristics in the development of computer-based adaptive learning environments. Computers in Human Behavior, 27 (1), 118–130. https://doi.org/10.1016/j.chb.2010.07.038. http://www.sciencedirect.com/science/article/pii/S0747563210002347.

    Article  Google Scholar 

  • Vanlehn, K. (2011). The relative effectiveness of human tutoring. Intelligent Tutoring Systems, and Other Tutoring Systems 46. https://doi.org/10.1080/00461520.2011.611369.

  • Werbach, K., & Hunter, D. (2012). For the win : how game thinking can revolutionize your business. Wharton.

  • Woolf, B.P. (2007). Building intelligent interactive tutors: student-centered strategies for revolutionizing e-learning. San Francisco: Morgan Kaufmann Publishers Inc.

    Google Scholar 

  • Xhakaj, F., Aleven, V., & McLaren, B.M. (2017). Effects of a teacher dashboard for an intelligent tutoring system on teacher knowledge, lesson planning, lessons and student learning. In E. Lavoue, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.) Data driven approaches in digital education (pp. 315–329). Cham: Springer International Publishing.

  • Zaric, N., Gottschlich, M., Roepke, R., & Schroeder, U. (2020). Supporting gamification with an interactive gamification analytics tool (igat). In C. Alario-hoyos, M.J. Rodríguez-Triana, M. Scheffel, I. Arnedillo-Sánchez, & S.M. Dennerlein (Eds.) Addressing global challenges and quality education (pp. 461–466). Cham: Springer International Publishing.

  • Zatarain-Cabada, R., Barrón-Estrada, M.L., & Ríos-Félix, J.M. (2017). Affective learning system for algorithmic logic applying gamification. In O. Pichardo-Lagunas S. Miranda-Jiménez (Eds.) Advances in soft computing (pp. 536–547). Cham: Springer International Publishing.

  • Zimmerman, J., & Forlizzi, J. (2017). Speed dating: Providing a menu of possible futures. She Ji, 3(1), 30–50. https://doi.org/10.1016/j.sheji.2017.08.003.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computing Institute, Federal University of Alagoas, 57072-900, Maceió, AL, Brazil

    Kamilla Tenório, Mateus Monteiro, Aristoteles Peixoto & Alan Pedro da Silva

  2. School of Medicine, Federal University of Alagoas, 57072-900, Maceió, AL, Brazil

    Diego Dermeval

Authors
  1. Kamilla Tenório
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Diego Dermeval
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mateus Monteiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aristoteles Peixoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alan Pedro da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Diego Dermeval.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tenório, K., Dermeval, D., Monteiro, M. et al. Exploring Design Concepts to Enable Teachers to Monitor and Adapt Gamification in Adaptive Learning Systems: A Qualitative Research Approach. Int J Artif Intell Educ 32, 867–891 (2022). https://doi.org/10.1007/s40593-021-00274-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40593-021-00274-y

Keywords

Search

Navigation