Abstract
In an effort to modernize training, the United States Navy and Marine Corps have placed an emphasis on identifying effective, learner-centric instructional methods. One avenue is to apply individualized training techniques, such as adaptive sequencing to flashcard-based study, a popular tool used for independent study. Therefore, the goal of this research was to compare two adaptive sequencing methods to identify the most efficient and effective approach for long-term retention. Participants learned armored vehicle identification in one of three conditions, 1) Adaptive Response-Time Based-Sequencing (ARTS), which uses accuracy and reaction time to prioritize flashcards adaptively, 2) Leitner, which uses accuracy to create decks and prioritize flashcards adaptively, and 3) Random, which was the control condition that sequenced the flashcards randomly. We found no differences between the three conditions in terms of learning efficiency and delayed learning gains. However, we found that participants in the Leitner condition completed training significantly faster those in the other conditions. After controlling for in-training measures, we found participants in the Leitner condition had the lowest delayed learning gains and there were no significant differences between the ARTS and Random conditions. These data suggest that the “efficiency” associated with the Leitner condition translated to the worst long-term retention. Additionally, neither adaptive approach outperformed the random sequencing condition, suggesting that random sequencing may provide its own spacing due to the number of cards in the deck. More research is needed to determine whether adaptive sequencing provides added value as an adaptive approach for single-session flashcard study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahrick, H.P., Bahrick, L.E., Bahrick, A.S., Bahrick, P.E.: Maintenance of foreign language vocabulary and the spacing effect. Psychol. Sci. 4(5), 316–321 (1993)
Berger, D.H.: 38th Commandant’s Planning Guidance (SSIC No. 05000 General Admin & Management) (2019). https://www.marines.mil/News/Publications/MCPEL/Electronic-Library-Display/Article/1907265/38th-commandants-planning-guidance-cpg/
Bjork, R.A., Dunlosky, J., Kornell, N.: Self-regulated learning: beliefs, techniques, and illusions. Annu. Rev. Psychol. 64, 417–444 (2013)
Carrier, M., Pashler, H.: The influence of retrieval on retention. Mem. Cognit. 20(6), 633–642 (1992). https://doi.org/10.3758/BF03202713
Cepeda, N.J., Pashler, H., Vul, E., Wixted, J.T., Rohrer, D.: Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychol. Bull. 132(3), 354–380 (2006)
Ebbinghaus, H.: Über das Gedächtnis: Untersuchungen zur Experimentellen Psychologie (About Memory: Studies on Experimental Psychology). Duncker & Humblot (1885)
Ebbinghaus, H.: Memory: a contribution to experimental psychology. In: Ruger, H.A., Bussenius, C.E., Hiligard, E.R. (ets.) Trans. Dover Publications. (Original work published in 1885) (1964)
Gilday, M.M.: Fragmentary order: a design for maintaining maritime superiority (2019). https://www.navy.mil/DesktopModules/ArtithecleCS/Print.aspx?PortalId=1&ModuleId=685&Article=2237608
Glenberg, A.M., Lehmann, T.S.: Spacing repetitions over 1 week. Mem. Cognit. 8(6), 528–538 (1980). https://doi.org/10.3758/BF03213772
Gromada, J.: The Leitner system: how does it work? Mindedge (2021)
https://www.mindedge.com/learning-science/the-leitner-system-how-does-it-work/
Hake, R.R.: Interactive-engagement versus traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses. Am. J. Phys. 66(1), 64–74 (1998)
Hartwig, M.K., Dunlosky, J.: Study strategies of college students: Are self-testing and scheduling related to achievement? Psychon. Bull. Rev. 19(1), 126–134 (2012). https://doi.org/10.3758/s13423-011-0181-y
Karpicke, J.D., Butler, A.C., Roediger, H.L., III.: Metacognitive strategies in student learning: do students practise retrieval when they study on their own? Memory 17(4), 471–479 (2009)
Karpicke, J.D., Roediger, H.L., III.: The critical importance of retrieval for learning. Science 319(5865), 966–968 (2008)
Kornell, N.: Optimising learning using flashcards: spacing is more effective than cramming. Appl. Cogn. Psychol. 23(9), 1297–1317 (2009)
Kornell, N., Bjork, R.A.: The promise and perils of self-regulated study. Psychon. Bull. Rev. 14(2), 219–224 (2007). https://doi.org/10.3758/BF03194055
Kornell, N., Bjork, R.A.: Learning concepts and categories: Is spacing the “enemy of induction”? Psychol. Sci. 19(6), 585–592 (2008a)
Kornell, N., Bjork, R.A.: Optimising self-regulated study: the benefits—and costs—of dropping flashcards. Memory 16(2), 125–136 (2008b)
Landsberg, C.R., Astwood, R.S., Jr., Van Buskirk, W.L., Townsend, L.N., Steinhauser, N.B., Mercado, A.D.: Review of adaptive training system techniques. Mil. Psychol. 24(2), 96–113 (2012)
Landsberg, C.R., Van Buskirk, W.L., Astwood, R.S., Mercado, A.D., Aakre, A.J.: Adaptive training considerations for simulation-based training systems (Special report 2010–001). Defense Technical Information Center (DTIC) (2011)
Leitner, S.: So lernt man lernen. AngewandteLernpsychologie – ein Weg zum Erfolg. Herder (1972)
Marraffino, M.D., Johnson, C.I., Whitmer, D.E., Steinhauser, N.B., Clement, A.: Advise when ready for game plan: Adaptive training for JTACs. In: Proceedings of the Interservice/Industry Training, Simulation and Education Conference (2019)
Marraffino, M.D., Schroeder, B.L., Fraulini, N.W., Van Buskirk, W.L., Johnson, C.I.: Adapting training in real time: an empirical test of adaptive difficulty schedules. Mil. Psychol. 33(3), 136–151 (2021)
Mettler, E., Burke, T., Massey, C.M., Kellman, P.J.: Comparing adaptive and random spacing schedules during learning to mastery criteria. In: Proceedings of the 42nd Annual Conference of the Cognitive Science Society, pp. 773–779 (2020)
Mettler, E., Kellman, P.J.: Adaptive response-time-based category sequencing in perceptual learning. Vision. Res. 99, 111–123 (2014)
Mettler, E., Massey, C.M., Kellman, P.J.: Improving adaptive learning technology through the use of response times. In: Proceedings of the 33rd Annual Conference of the Cognitive Science Society, pp. 2532–2537 (2011)
Mettler, E., Massey, C.M., Kellman, P.J.: A comparison of adaptive and fixed schedules of practice. J. Exp. Psychol. Gen. 145(7), 897–917 (2016)
Paivio, A.: Spacing of repetitions in the incidental and intentional free recall of pictures and words. J. Verbal Learn. Verbal Behav. 13(5), 497–511 (1974)
Peirce, N., Wade, V.: Personalised learning for casual games: The “Language Trap” online language learning game. In: Meyer, B. (Ed.) Proceedings of the 4th European Conference on Games Based Learning (ECGBL 2010), pp. 306–315. Academic Publishing (2010)
Pham, X.L., Chen, G.D., Nguyen, T.H., Hwang, W.Y.: Card-based design combined with spaced repetition: a new interface for displaying learning elements and improving active recall. Comput. Educ. 98, 142–156 (2016)
Pyc, M.A., Rawson, K.A.: Costs and benefits of dropout schedules of test–restudy practice: Implications for student learning. Appl. Cogn. Psychol. 25(1), 87–95 (2011)
Roediger, H.L., III., Butler, A.C.: The critical role of retrieval practice in long-term retention. Trends Cogn. Sci. 15(1), 20–27 (2011)
Roediger, H.L., III., Karpicke, J.D.: Test-enhanced learning: Taking memory tests improves long-term retention. Psychol. Sci. 17(3), 249–255 (2006)
Romero, C., Ventura, S., Gibaja, E.L., Hervás, C., Romero, F.: Web-based adaptive training simulator system for cardiac life support. Artif. Intell. Med. 38(1), 67–78 (2006)
Schuetze, U.: Spacing techniques in second language vocabulary acquisition: Short-term gains vs. long-term memory. Lang. Teach. Res. 19(1), 28–42 (2015)
Settles, B.: How we learn how you learn. Duolingo Blog (2016). https://blog.duolingo.com/how-we-learn-how-you-learn/
Shaughnessy, J.J., Zimmerman, J., Underwood, B.J.: The spacing effect in the learning of word pairs. Mem. Cognit. 2(4), 742–774 (1974). https://doi.org/10.3758/BF03198150
Shiffrin, R.M., Schneider, W.: Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. Psychol. Rev. 84(2), 127–190 (1977)
Sommer, W.G.: Procrastination and cramming: how adept students ace the system. J. Am. Coll. Health 39, 5–10 (1990)
Sternberg, S.: Memory-scanning: mental processes revealed by reaction-time experiments. Am. Sci. 57(4), 421–457 (1969)
Toppino, T.C., Cohen, M.S.: The testing effect and the retention interval: questions and answers. Exp. Psychol. 56(4), 252–257 (2009)
Van Buskirk, W.L., Fraulini, N.W., Schroeder, B.L., Johnson, C.I., Marraffino, M.D.: Application of theory to the development of an adaptive training system for a submarine electronic warfare task. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2019. LNCS, vol. 11597, pp. 352–362. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22341-0_28
VanLehn, K., et al.: The Andes physics tutoring system: lessons learned. Int. J. Artif. Intell. Educ. 15(3), 147–204 (2005)
Vaughn, K.E., Rawson, K.A.: Diagnosing criterion-level effects on memory: What aspects of memory are enhanced by repeated retrieval? Psychol. Sci. 22(9), 1127–1131 (2011)
Whelan, J.: Using the Leitner system to improve your study. Medium (2019)
https://jessewhelan.medium.com/using-the-leitner-system-to-improve-your-study-d5edafae7f0
Whitmer, D.E., Johnson, C.I., Marraffino, M.D., Hovorka, J.: Using adaptive flashcards for automotive maintenance training in the wild. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2021. LNCS, vol. 12792, pp. 466–480. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77857-6_33
Whitmer, D.E., Johnson, C.I., Marraffino, M.D., Pharmer, R.L., Blalock, L.D.: A mastery approach to flashcard-based adaptive training. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2020. LNCS, vol. 12214, pp. 555–568. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50788-6_41
Wissman, K.T., Rawson, K.A., Pyc, M.A.: How and when do students use flashcards? Memory 20(6), 568–579 (2012)
Acknowledgments
We gratefully acknowledge Dr. Peter Squire and the Office of Naval Research who sponsored this work (Funding Doc# N0001421WX00349). Presentation of this material does not constitute or imply its endorsement, recommendation, or favoring by the U.S. Navy or the Department of Defense (DoD). The opinions of the authors expressed herein do not necessarily state or reflect those of the U.S. Navy of DoD.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Whitmer, D.E., Johnson, C.I., Marraffino, M.D. (2022). Examining Two Adaptive Sequencing Approaches for Flashcard Learning: The Tradeoff Between Training Efficiency and Long-Term Retention. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2022. Lecture Notes in Computer Science, vol 13332. Springer, Cham. https://doi.org/10.1007/978-3-031-05887-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-05887-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05886-8
Online ISBN: 978-3-031-05887-5
eBook Packages: Computer ScienceComputer Science (R0)