Abstract
Although many praise the positive benefits of game-based training to increase learner engagement and performance, there has been little empirical research to support these claims. The goal of this experiment was to establish whether adding game features has a positive impact on performance during training and leads to better learning outcomes. Specifically, we explored whether the presence of game features (i.e., performance gauges) and competition features (i.e., leaderboard) affected motivation and learning outcomes within the Periscope Operator Adaptive Trainer (POAT). We conducted an experiment with 49 Submarine Officer Basic Course students who were assigned randomly to either training with a version of POAT with game features (Game Features condition) or one without game features (Control condition). Analyses revealed no differences between the two conditions on learning gains or reported motivation. The results did show that students in both conditions improved significantly on the accuracy (i.e., angle on the bow and range) and timeliness of their periscope calls from pre-test to post-test, providing additional support for the benefits of adaptive training but not game features.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The data were also submitted to a repeated measures ANOVA with training condition as the between-subjects variable and test type (pre- and post-test scores) as the within-subjects variable. The results indicated that AOB accuracy improved for both groups between the pre- and post-test [F(1,47) = 11.456, p = .001, ηp2 = .196], but there was no difference between training groups [F(1,47) = 0.192, p = .663, ηp2 = .004], nor was there an interaction between variables [F(1,47) = 1.573, p = .216, ηp2 = .032]. Overall, adaptive training increased AOB accuracies by approximately 50%.
- 2.
A second repeated measures ANOVA was conducted on pre- and post-test range scores. As with AOB, the accuracy for range scores also improved after adaptive training [F(1,47) = 20.846, p < .001, ηp2 = .307]. Again, the training groups did not differ on range accuracy [F(1,47) = 0.052, p = .821, ηp2 = .001], nor was there an interaction [F(1,47) = 0.257, p = .615, ηp2 = .005].
References
Bedwell, W.L., Pavlas, D., Heyne, K., Lazzara, E.H., Salas, E.: Toward a taxonomy linking game attributes to learning: An empirical study. Simul. Gaming 43(6), 729–760 (2012)
Cagiltay, N.E., Ozcelik, E., Ozcelik, N.S.: The effect of competition on learning in games. Comput. Educ. 87, 35–41 (2015)
Cameron, J., Pierce, W.D., Banko, K.M., Gear, A.: Achievement-based rewards and intrinsic motivation: a test of cognitive mediators. J. Educ. Psychol. 97(4), 641–655 (2005)
Ekstrom, R.B., French, J.W., Harman, H.H.: Manual for Kit of Factor-Referenced Cognitive Tests. Educational Testing Service, Princeton (1976)
Garcia, S.M., Tor, A., Gonzalez, R.: Ranks and rivals: a theory of competition. Pers. Soc. Psychol. Bull. 32(7), 970–982 (2006)
Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: a research and practice model. Simul. Gaming 33, 441–467 (2002)
Gee, J.P.: What Video Games have to Teach Us About Learning and Literacy. Pelgrave Macmillan, New York (2003)
Hays, R.T.: The effectiveness of instructional games: a literature review and discussion. Technical report, 2005–004, Naval Air Warfare Center Training Systems Division, Orlando, FL (2005)
Hawlitschek, A., Joeckel, S.: Increasing the effectiveness of digital educational games: the effects of a learning instruction on students’ learning, motivation and cognitive load. Comput. Hum. Behav. 72, 79–86 (2017)
Landsberg, C.R., Mercado, A., Van Buskirk, W.L., Lineberry, M., Steinhauser, N.: Evaluation of an adaptive training system for submarine periscope operations. In: Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting, pp. 2422–2426. SAGE Publications, Los Angeles (2012)
Lister, M., College, M.: Gamification: the effect on student motivation and performance at the post-secondary level. Issues Trends Educ. Technol. 3(2), 1–22 (2015)
Marraffino, M.D., Johnson, C.I., Whitmer, D.E., Steinhauser, N.B., Clement, A.: Advise when ready for game plan: adaptive training for JTACs. In: Proceedings of the Interservice/Industry, Training, Simulation, and Education Conference (2019)
Mayer, R.E.: Computer games in education. Ann. Rev. Psychol. 70, 22.1–22.19 (2019)
Mayer, R.E., Johnson, C.I.: Adding instructional features that promote learning in a game-like environment. J. Educ. Comput. Res. 42, 241–265 (2010)
O’Neil, H.F., Perez, R.S. (eds.): Computer Games and Team and Individual Learning. Elsevier, Oxford (2008)
Plass, J.L., Homer, B.D., Kinzer, C.K.: Foundations of game-based learning. Educ. Psychol. 50(4), 258–283 (2015)
Plass, J.L., et al.: The impact of individual, competitive, and collaborative mathematics game play on learning, performance, and motivation. J. Educ. Psychol. 105(4), 1050–1066 (2013)
Prensky, M.: Digital Game-Based Learning. McGraw-Hill, New York (2001)
Ray, J.J.: A quick measure of achievement motivation: validated in Australia and reliable in Britain and South Africa. Aust. Psychol. 14(3), 337–344 (1979)
Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78 (2000)
Shute, V.J., Ventura, M., Kim, Y.J.: Assessment and learning of qualitative physics in newton’s playground. J. Educ. Res. 106(6), 423–430 (2013)
Sitzmann, T.: A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Pers. Psychol. 64, 489–528 (2011)
Van Buskirk, W.L., Fraulini, N.W., Schroeder, B.L., Johnson, C.I., Marraffino, M.D.: Application of theory to the development of an adaptive training system for a submarine electronic warfare task. In: Sottilare, R.A., Schwarz, J. (eds.) HCII 2019. LNCS, vol. 11597, pp. 352–362. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22341-0_28
Van Buskirk, W.L., Steinhauser, N.B., Mercado, A.D., Landsberg, C.R., Astwood, R.S.: A comparison of the micro-adaptive and hybrid approaches to adaptive training. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 58(1), pp. 1159–1163. SAGE Publications, Los Angeles, September 2014
van Merrienboer, J.J., Sweller, J.: Cognitive load theory and complex learning: recent developments and future directions. Educ. Psychol. Rev. 17(2), 147–177 (2005)
Vandercruysse, S., Vandewaetere, M., Cornillie, F., Clarebout, G.: Competition and students’ perceptions in a game-based language learning environment. Educ. Tech. Res. Dev. 61(6), 927–950 (2013). https://doi.org/10.1007/s11423-013-9314-5
Wang, H., Sun, C.T.: Game reward systems: Gaming experiences and social meanings. In: DiGRA Conference, September 2011
Wouters, P., van Nimwegen, C., van Oostendorp, H., van der Speck, E.: A meta-analysis of the cognitive and motivational effects of serious games. J. Educ. Psychol. 105, 249–265 (2013)
Acknowledgments
We gratefully acknowledge Dr. James Sheehy who sponsored this work through the Section 219 Naval Innovative Science and Engineering Basic and Applied Research program. We would also like to thank Derek Tolley developing the versions of POAT used in the experiment. Presentation of this material does not constitute or imply its endorsement, recommendation, or favoring by the U.S. Navy or Department of Defense (DoD). The opinions of the authors expressed herein do not necessarily state or reflect those of the U.S. Navy or DoD.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply
About this paper
Cite this paper
Johnson, C.I., Bailey, S.K.T., Mercado, A.D. (2020). Does Gamification Work? Analyzing Effects of Game Features on Learning in an Adaptive Scenario-Based Trainer. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2020. Lecture Notes in Computer Science(), vol 12214. Springer, Cham. https://doi.org/10.1007/978-3-030-50788-6_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-50788-6_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50787-9
Online ISBN: 978-3-030-50788-6
eBook Packages: Computer ScienceComputer Science (R0)