Skip to main content
SpringerLink
Log in

Not All Pain Leads to Gain: The Role of Learner Engagement in Adaptive Flashcard Training

  • Conference paper
  • First Online:
Adaptive Instructional Systems (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14044))

Included in the following conference series:

Abstract

Adaptive training methods have been designed to enhance students’ learning outcomes by tailoring the educational content based on the learner’s performance during training. In the present study, we examined different adaptive sequencing methods for a flashcard-based trainer. One sequencing method, the Adaptive Response Time-based Sequencing (ARTS) algorithm presents cards based on an individual learner’s accuracy and reaction time, such that incorrectly identified cards are prioritized over correctly identified cards. Although previous research has suggested that ARTS is more efficient and effective than other forms of flashcard sequencing, recent research was unable to replicate these findings. To that end, the current experiment compared ARTS to an adaptive control condition that reversed the ARTS algorithm and investigated if learner engagement plays a role in adaptive flashcard-based training. A sample of 50 college students learned to identify African countries in one of two adaptive flashcard sequencing conditions – ARTS and control. Engagement was measured using the flow state scale for occupational tasks and training effectiveness was determined by calculating immediate and delayed learning gains. Results revealed no statistically significant differences between ARTS and the control on immediate and delayed gains. Further, the ARTS group reported significantly lower engagement levels than the control group. A mediation analysis revealed that the relationship between the training and the learning gains was significantly mediated by engagement in an inverse format, suggesting that training reduced the levels of engagement which in turn canceled out the learning gains. Based on these findings, we present the theoretical and practical implications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Whitmer, D.E., Johnson, C.I., Marraffino, M.D.: Examining two adaptive sequencing approaches for flashcard learning: the Tradeoff between training efficiency and long-term retention. In: Sottilare, R.A., Schwarz, J. (eds.) Adaptive Instructional Systems. HCII 2022. Lecture Notes in Computer Science, vol. 13332, pp. 126–139. Springer, Cham (2022) https://doi.org/10.1007/978-3-031-05887-5_10

  2. Landsberg, C.R., Astwood, R.S., Van Buskirk, W.L., Townsend, L.N., Steinhauser, N.B., Mercado, A.D.: Review of adaptive training system techniques. Mil. Psychol. 24(2), 96–113 (2012). https://doi.org/10.1080/08995605.2012.672903

    Article  Google Scholar 

  3. Mettler, E., Burke, T., Massey, C.M., Kellman, P.J.: Comparing adaptive and random spacing schedules during learning to mastery criteria. In: CogSci Annual Conference of the Cognitive Science Society. Cognitive Science Society (US). Conference, vol. 2020, pp. 773–779 (2020)

    Google Scholar 

  4. Mettler, E., Kellman, P.J.: Adaptive response-time-based category sequencing in perceptual learning. Vision Res. 99, 111–123 (2014). https://doi.org/10.1016/j.visres.2013.12.009

    Article  Google Scholar 

  5. Whitmer, D.E., Johnson, C.I., Marraffino, M.D., Pharmer, R.L., Blalock, L.D.: A mastery approach to flashcard-based adaptive training. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2020. LNCS, vol. 12214, pp. 555–568. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50788-6_41

    Chapter  Google Scholar 

  6. Zahabi, M., Abdul Razak, A.M.: Adaptive virtual reality-based training: a systematic literature review and framework. Virtual Reality 24(4), 725–752 (2020). https://doi.org/10.1007/s10055-020-00434-w

    Article  Google Scholar 

  7. Durlach, P.J., Ray, J.M.: Designing Adaptive Instructional Environments: Insights from Empirical Evidence. Defense Technical Information Center, Fort Belvoir (2011). https://doi.org/10.21236/ADA552677

  8. Graesser, A.C., D’Mello, S.: Chapter five - emotions during the learning of difficult material. Psychol. Learn. Motiv. 57, 183–225 (2012). https://doi.org/10.1016/B978-0-12-394293-7.00005-4

    Article  Google Scholar 

  9. Holmes, J., Gathercole, S.E., Dunning, D.L.: Adaptive training leads to sustained enhancement of poor working memory in children. Dev. Sci. 12(4), F9–F15 (2009). https://doi.org/10.1111/j.1467-7687.2009.00848.x

    Article  Google Scholar 

  10. Landsberg, C.R., Mercado, A.D., Van Buskirk, W.L., Lineberry, M., Steinhauser, N.: Evaluation of an adaptive training system for submarine periscope operations. In: Proceedings of the Human Factors and Ergonomics Society annual meeting, vol. 56, no. 1, pp. 2422–2426 (2012)

    Google Scholar 

  11. Mettler, E., Massey, C.M., Kellman, P.J.: Improving adaptive learning technology through the use of response times (2011). https://eric.ed.gov/?id=ED549436. Accessed 28 Jan 2023

  12. Engeser, S., Rheinberg, F.: Flow, performance and moderators of challenge-skill balance. Motiv. Emot. 32(3), 158–172 (2008)

    Article  Google Scholar 

  13. Shernoff, D.J., Csikszentmihalyi, M., Shneider, B., Shernoff, E.S.: Student engagement in high school classrooms from the perspective of flow theory. Sch. Psychol. Q. 18, 158–176 (2003). https://doi.org/10.1521/scpq.18.2.158.21860

    Article  Google Scholar 

  14. Alyüz, N., et al.: Towards an emotional engagement model: can affective states of a learner be automatically detected in a 1: 1 learning scenario?. In: UMAP (Extended Proceedings) (2016)

    Google Scholar 

  15. Carroll, M., Lindsey, S., Chaparro, M., Winslow, B.: An applied model of learner engagement and strategies for increasing learner engagement in the modern educational environment. Interact. Learn. Environ. 29(5), 757–771 (2021)

    Article  Google Scholar 

  16. de Manzano, Ö., Theorell, T., Harmat, L., Ullén, F.: The psychophysiology of flow during piano playing. Emotion 10, 301–311 (2010). https://doi.org/10.1037/a0018432

    Article  Google Scholar 

  17. Mark, G., Czerwinski, M., Iqbal, S.T.: Effects of individual differences in blocking workplace distractions. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, pp. 1–12 (2018). https://doi.org/10.1145/3173574.3173666

  18. D’Mello, S.: A selective meta-analysis on the relative incidence of discrete affective states during learning with technology. J. Educ. Psychol. 105(4), 1082 (2013)

    Article  Google Scholar 

  19. Frenzel, A.C., Pekrun, R., Goetz, T.: Perceived learning environment and students’ emotional experiences: a multilevel analysis of mathematics classrooms. Learn. Instr. 17(5), 478–493 (2007)

    Article  Google Scholar 

  20. Goetz, T., Hall, N.C.: Emotion and achievement in the classroom. In: International Guide to Student Achievement, Routledge, pp. 192–195 (2013)

    Google Scholar 

  21. Cronhjort, M., Filipsson, L., Weurlander, M.: Improved engagement and learning in flipped-classroom calculus. Teach. Math. Appl. Int. J. IMA 37(3), 113–121 (2018). https://doi.org/10.1093/teamat/hrx007

    Article  Google Scholar 

  22. Guo, Y.: The influence of academic autonomous motivation on learning engagement and life satisfaction in adolescents: the mediating role of basic psychological needs satisfaction. J. Educ. Learn. 7(4), 254 (2018). https://doi.org/10.5539/jel.v7n4p254

    Article  Google Scholar 

  23. Erhel, S., Jamet, E.: Improving instructions in educational computer games: exploring the relations between goal specificity, flow experience and learning outcomes. Comput. Hum. Behav. 91, 106–114 (2019). https://doi.org/10.1016/j.chb.2018.09.020

    Article  Google Scholar 

  24. Hou, H.-T.: Integrating cluster and sequential analysis to explore learners’ flow and behavioral patterns in a simulation game with situated-learning context for science courses: a video-based process exploration. Comput. Hum. Behav. 48, 424–435 (2015)

    Article  Google Scholar 

  25. Leiker, A.M., Miller, M., Brewer, L., Nelson, M., Siow, M., Lohse, K.: The relationship between engagement and neurophysiological measures of attention in motion-controlled video games: a randomized controlled trial. JMIR Serious Games 4(1), e5460 (2016). https://doi.org/10.2196/games.5460

    Article  Google Scholar 

  26. Christenson, S., Reschly, A.L., Wylie, C.: Handbook of Research on Student Engagement, vol. 840. Springer, New York (2012) https://doi.org/10.1007/978-1-4614-2018-7

  27. Finn, J.D., Zimmer, K.S.: Student Engagement: What is it? Why Does it Matter?. Handbook of Research on Student Engagement, pp. 97–131 (2012)

    Google Scholar 

  28. Pardos, Z.A., Baker, R.S., San Pedro, M.O., Gowda, S.M., Gowda, S.M.: Affective states and state tests: investigating how affect and engagement during the school year predict end-of-year learning outcomes. J. Learn. Anal. 1(1), 107–128 (2014)

    Article  Google Scholar 

  29. Loderer, K., Pekrun, R., Lester, J.C.: Beyond cold technology: a systematic review and meta-analysis on emotions in technology-based learning environments. Learn. Instr. 70, 101162 (2020). https://doi.org/10.1016/j.learninstruc.2018.08.002

    Article  Google Scholar 

  30. Mettler, E., Massey, C.M., Kellman, P.J.: A comparison of adaptive and fixed schedules of practice. J. Exp. Psychol. Gen. 145(7), 897 (2016)

    Article  Google Scholar 

  31. Bahrick, H.P., Bahrick, L.E., Bahrick, A.S., Bahrick, P.E.: Maintenance of foreign language vocabulary and the spacing effect. Psychol. Sci. 4(5), 316–321 (1993)

    Article  Google Scholar 

  32. Cepeda, N.J., Pashler, H., Vul, E., Wixted, J.T., Rohrer, D.: Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychol. Bull. 132(3), 354 (2006)

    Article  Google Scholar 

  33. Kornell, N.: Optimising learning using flashcards: spacing is more effective than cramming. Appl. Cogn. Psychol. Off. J. Soc. Appl. Res. Mem. Cogn. 23(9), 1297–1317 (2009)

    Google Scholar 

  34. Pyc, M.A., Rawson, K.A.: Testing the retrieval effort hypothesis: does greater difficulty correctly recalling information lead to higher levels of memory? J. Mem. Lang. 60(4), 437–447 (2009)

    Article  Google Scholar 

  35. Mettler, E., Massey, C.M., Burke, T., Garrigan, P., Kellman, P.J.: ‘Enhancing adaptive learning through strategic scheduling of passive and active learning modes. In: Proceedings of the 40th Annual Conference of the Cognitive Science Society (2018). https://par.nsf.gov/biblio/10087104-enhancing-adaptive-learning-through-strategic-scheduling-passive-active-learning-modes. Accessed 03 Feb 2023

  36. Appleton, J.J., Christenson, S.L., Kim, D., Reschly, A.L.: Measuring cognitive and psychological engagement: validation of the student engagement instrument. J. Sch. Psychol. 44(5), 427–445 (2006). https://doi.org/10.1016/j.jsp.2006.04.002

    Article  Google Scholar 

  37. Wang, C., Mirzaei, T., Xu, T., Lin, H.: How learner engagement impacts non-formal online learning outcomes through value co-creation: an empirical analysis. Int. J. Educ. Technol. High. Educ. 19(1), 1–26 (2022)

    Article  Google Scholar 

  38. Lee, J., Park, T., Davis, R.O.: What affects learner engagement in flipped learning and what predicts its outcomes? Br. J. Educ. Technol. 53(2), 211–228 (2022). https://doi.org/10.1111/bjet.12717

    Article  Google Scholar 

  39. Moore, K.A., Lippman, L.H.: What do children need to flourish?: Conceptualizing and measuring indicators of positive development, vol. 3. Springer, New York (2006). https://doi.org/10.1007/b100487

  40. Sinatra, G.M., Heddy, B.C., Lombardi, D.: The challenges of defining and measuring student engagement in science. Educ. Psychol. 50(1), 1–13 (2015)

    Article  Google Scholar 

  41. Nakamura, J., Csikszentmihalyi, M.: The Concept of Flow. Handbook of Positive Psychology, vol. 89, p. 105 (2002)

    Google Scholar 

  42. Whitson, C., Consoli, J.: Flow theory and student engagement. J. Cross-Discip. Perspect. Educ. 2(1), 40–49 (2009)

    Google Scholar 

  43. Shernoff, D.J., Anderson, B.: Enacting Flow and Student Engagement in the College Classroom. The Wiley Blackwell Handbook of Positive Psychological Interventions, pp. 194–212 (2014)

    Google Scholar 

  44. Shernoff, D.J., Csikszentmihalyi, M.: Cultivating Engaged Learners and Optimal Learning Environments’, Handbook of Positive Psychology in Schools, vol. 131, p. 145 (2009)

    Google Scholar 

  45. Csikszentmihalyi, M.: Flow and education. NAMTA J. 22(2), 2–35 (1997)

    Google Scholar 

  46. Pavlas, D., Heyne, K., Bedwell, W., Lazzara, E., Salas, E.: Game-based learning: the impact of flow state and videogame self-efficacy. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 54, no. 28, pp. 2398–2402 (2010)

    Google Scholar 

  47. Jackson, S.A., Marsh, H.W.: Development and validation of a scale to measure optimal experience: the flow state scale. J. Sport Exerc. Psychol. 18(1), 17–35 (1996)

    Article  Google Scholar 

  48. Yoshida, K., Sawamura, D., Inagaki, Y., Ogawa, K., Ikoma, K., Sakai, S.: Brain activity during the flow experience: a functional near-infrared spectroscopy study. Neurosci. Lett. 573, 30–34 (2014)

    Article  Google Scholar 

  49. Yoshida, K., et al.: The flow state scale for occupational tasks: development, reliability, and validity. Hong Kong J. Occup. Ther. 23(2), 54–61 (2013)

    Article  Google Scholar 

  50. Bell, B., Kelsey, E., Nye, B.: Monitoring engagement and motivation across learning environments, Los Angel (2019)

    Google Scholar 

  51. Hayes, A.F.: Beyond Baron and Kenny: statistical mediation analysis in the new millennium. Commun. Monogr. 76(4), 408–420 (2009)

    Article  Google Scholar 

  52. Hayes, A.F., Preacher, K.J., Myers, T.A.: Mediation and the estimation of indirect effects in political communication research. Sourceb. Polit. Commun. Res. Methods, Measures, Anal. Tech.23(1), 434–465 (2011)

    Google Scholar 

  53. Iacobucci, D.: Mediation analysis and categorical variables: the final frontier. J. Consum. Psychol. 22(4), 582–594 (2012)

    Article  Google Scholar 

  54. Hu, P.J.-H., Hui, W.: Examining the role of learning engagement in technology-mediated learning and its effects on learning effectiveness and satisfaction. Decis. Support Syst. 53(4), 782–792 (2012). https://doi.org/10.1016/j.dss.2012.05.014

    Article  Google Scholar 

  55. Nkhoma, M., Sriratanaviriyakul, N., Pham Cong, H., Khai Lam, T.: Examining the mediating role of learning engagement, learning process and learning experience on the learning outcomes through localized real case studies. Educ. Train. 56(4), 287–302 (2014). https://doi.org/10.1108/ET-01-2013-0005

    Article  Google Scholar 

  56. Ayçiçek, B., Yanpar Yelken, T.: The Effect of flipped classroom model on students’ classroom engagement in teaching English. Int. J. Instr. 11(2), 385–398 (2018)

    Google Scholar 

  57. Bond, M.: Facilitating student engagement through the flipped learning approach in K-12: a systematic review. Comput. Educ. 151, 103819 (2020). https://doi.org/10.1016/j.compedu.2020.103819

    Article  Google Scholar 

  58. Johnson, L.S.: Relationship of instructional methods to student engagement in two public high schools. Am. Second. Educ. 36(2), 69–87 (2008)

    Google Scholar 

  59. Hamari, J., Koivisto, J.: Measuring flow in gamification: dispositional flow scale-2. Comput. Hum. Behav. 40, 133–143 (2014). https://doi.org/10.1016/j.chb.2014.07.048

    Article  Google Scholar 

  60. Shernoff, D.J.: Optimal Learning Environments to Promote Student Engagement (2013)

    Google Scholar 

  61. Procci, K., Singer, A.R., Levy, K.R., Bowers, C.: Measuring the flow experience of gamers: an evaluation of the DFS-2. Comput. Hum. Behav. 28(6), 2306–2312 (2012)

    Article  Google Scholar 

  62. Wang, L., Chen, M.: The effects of game strategy and preference-matching on flow experience and programming performance in game-based learning. Innov. Educ. Teach. Int. 47(1), 39–52 (2010). https://doi.org/10.1080/14703290903525838

    Article  Google Scholar 

  63. Ceja, L., Navarro, J.: “Suddenly I get into the zone”: examining discontinuities and nonlinear changes in flow experiences at work. Hum. Relat. 65(9), 1101–1127 (2012). https://doi.org/10.1177/0018726712447116

    Article  Google Scholar 

  64. Oyama, Y., Manalo, E., Nakatani, Y.: The Hemingway effect: how failing to finish a task can have a positive effect on motivation. Think. Ski. Creat. 30, 7–18 (2018). https://doi.org/10.1016/j.tsc.2018.01.001

    Article  Google Scholar 

  65. Fullagar, C.J., Knight, P.A., Sovern, H.S.: Challenge/skill balance, flow, and performance anxiety. Appl. Psychol. 62(2), 236–259 (2013). https://doi.org/10.1111/j.1464-0597.2012.00494.x

    Article  Google Scholar 

  66. LePine, J.A., LePine, M.A., Jackson, C.L.: Challenge and hindrance stress: relationships with exhaustion, motivation to learn, and learning performance. J. Appl. Psychol. 89, 883–891 (2004). https://doi.org/10.1037/0021-9010.89.5.883

    Article  Google Scholar 

  67. Senko, C., Tropiano, K.L.: Comparing three models of achievement goals: goal orientations, goal standards, and goal complexes. J. Educ. Psychol. 108, 1178–1192 (2016). https://doi.org/10.1037/edu0000114

    Article  Google Scholar 

  68. Bloom, B.S.: Human characteristics and school learning. New York, NY, US: McGraw-Hill, pp. xii, 284 (1976)

    Google Scholar 

  69. Bloom, B.S.: The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. Educ. Res. 13(6), 4–16 (1984). https://doi.org/10.3102/0013189X013006004

    Article  Google Scholar 

  70. Vygotsky, L.S., Cole, M.: Mind in Society: Development of Higher Psychological Processes. Harvard University Press, Cambridge (1978)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Florida Institute of Technology, Melbourne, FL, 32901, USA

    Maureen Namukasa, Meredith Carroll, Whitney-Marie Igwe & Morgan Nash

  2. Naval Air Warfare Center Training Systems Division, Orlando, FL, 32825, USA

    Matthew D. Marraffino & Daphne E. Whitmer

  3. Quantum Improvements Consulting, LLC, Orlando, FL, 32803, USA

    Cheryl I. Johnson

Authors
  1. Maureen Namukasa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Meredith Carroll
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew D. Marraffino
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cheryl I. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daphne E. Whitmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Whitney-Marie Igwe
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Morgan Nash
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maureen Namukasa .

Editor information

Editors and Affiliations

  1. Soar Technology, Inc., Orlando, FL, USA

    Robert A. Sottilare

  2. Fraunhofer FKIE, Wachtberg, Germany

    Jessica Schwarz

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Namukasa, M. et al. (2023). Not All Pain Leads to Gain: The Role of Learner Engagement in Adaptive Flashcard Training. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2023. Lecture Notes in Computer Science, vol 14044. Springer, Cham. https://doi.org/10.1007/978-3-031-34735-1_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34735-1_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34734-4

  • Online ISBN: 978-3-031-34735-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation