Abstract
This study is a meta-analytical study that examines the effectiveness of gamification in learning performance in educational settings (n = 29; year-span = 2011–2019). Specifically, it aimed to investigate (a) whether gamification could improve learning performance, and (b) whether peer interaction (i.e., peer competition and peer collaboration) moderated the effectiveness of gamification in learning performance. Results from random-effects models showed significant effects of gamification in learning performance (g = .595, 95% CI [.432, .758], N = 3515). This effect remained robust after excluding outliers and was stable in a sub-split analysis that excludes studies with low methodological rigor (i.e., studies with pre-post test design). Subgroup analyses revealed a moderating effect of peer competition in gamification in learning, suggesting that competitive games were better than non-competitive games for promoting learning performance in educational settings. However, this effect was not robust and no evidence of subgroup differences were found in the sub-split analysis. Peer collaboration did not moderate the effectiveness of gamification in learning as no subgroup differences were found between collaborative games and non-collaborative games. The effectiveness of games that were both competitive and collaborative did not differ from those that were only competitive. Other moderators such as education level and research design were also investigated. No subgroup differences were found for these two moderators. Educational implications and limitations were further discussed.
Similar content being viewed by others
Notes
As the pre-post test correlation was required to compute the effect sizes of studies with pre-test measures (e.g., see Borenstein, 2009, p. 227, Equation 12.21), missing values of the correlation in some studies would lead to missing effect sizes of those studies. As the pre-post test correlations reported were inconsistent across studies (the average estimate is .634, range from .387 to .8), it was inappropriate to impute a common value from them (Higgins & Green, 2011). Therefore, a sensitivity analysis was conducted to evaluate the robustness of the results. Different estimates drawn from the aforementioned range of pre-post test correlations (.40, .60 and .8 0) were imputed for missing pre-post test correlations and no substantial change in results was found . Thus, we reported findings of an imputed estimate of .60 in this meta-analysis as this was approximate to the average
In subgroup analysis, CMA offers two options in terms of computational models: a pooled estimate of τ2 and with separate estimates of τ2. In this study, separate estimates of τ2 were used for all moderators since a) the true study-to-study dispersion was expected to differ across subgroups; b) there were unequal numbers of studies in each subgroup; and c) the variances in subgroups were likely to vary for moderators with intervention effects (e.g., competitive games versus non-competitive games) (Borenstein et al., 2009)
References
Abrantes, J. L., Seabra, C., & Lages, L. F. (2007). Pedagogical affect, student interest, and learning performance. Journal of Business Research, 60(9), 960–964.
Ames, C. (1984). Achievement attributions and self-instructions under competitive and individualistic goal structures. Journal of Educational Psychology, 76(3), 478–487.
Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, 100322–100341.
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 1088–1101.
Blohm, I., & Leimeister, J. M. (2013). Gamification. Wirtschaftsinformatik, 55(4), 275–278.
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. John Wiley & Sons.
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2014). Comprehensive meta-analysis (CMA) software. Version 3.0.
Briffa, M., Jaftha, N., Loreto, G., Pinto, F. C. M., Chircop, T., & Hill, C. (2020). Improved students’ performance within gamified learning environment: A meta-analysis study. International Journal of Education and Research, 8(1), 223–244.
Buckley, P., & Doyle, E. (2017). Individualising gamification: An investigation of the impact of learning styles and personality traits on the efficacy of gamification using a prediction market. Computers & Education, 106, 43–55.
Burguillo, J. C. (2010). Using game theory and competition-based learning to stimulate student motivation and performance. Computers & Education, 55(2), 566–575.
Cagiltay, N. E., Ozcelik, E., & Ozcelik, N. S. (2015). The effect of competition on learning in games. Computers & Education, 87, 35–41.
Chen, Z. H. (2014). Exploring students’ behaviors in a competition-driven educational game. Computers in Human Behavior, 35, 68–74.
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122.
Cole, M., John-Steiner, V., Scribner, S., & Souberman, E. (Eds.). (1978). L.S. Vygotsky: Mind in society: The development of higher psychological processes. Harvard University Press.
De Byl, P. (2013). Factors at play in tertiary curriculum gamification. International Journal of Game-Based Learning (IJGBL), 3(2), 1–21.
de Sousa Borges, S., Durelli, V. H., Reis, H. M., & Isotani, S. (2014, March). A systematic mapping on gamification applied to education. In proceedings of the 29th annual ACM symposium on applied computing (pp. 216-222). ACM.
Deci, E. L., & Ryan, R. M. (2002). Overview of self-determination theory: An organismic dialectical perspective. Handbook of self-determination research (pp. 3-33). University of Rochester Press.
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: Defining gamification. In proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (pp. 9-15). ACM.
Deutsch, M. (1962). Cooperation and trust: Some theoretical notes.
Dicheva, D., & Dichev, C. (2015, October). Gamification in education: Where are we in 2015?. In E-learn: World conference on E-learning in corporate, government, healthcare, and higher education (pp. 1445-1454). Association for the Advancement of computing in education (AACE).
Dillenbourg, P. (1999). What do you mean by 'collaborative learning'? Elsevier Science, 1–16.
Dillenbourg, P., & Fischer, F. (2007). Basics of computer-supported collaborative learning. Zeitschrift für Berufs- und Wirtschaftspädagogik., 21, 111–130.
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315(7109), 629–634.
Eseryel, D., Law, V., Ifenthaler, D., Ge, X., & Miller, R. (2014a). An investigation of the interrelationships between motivation, engagement, and complex problem solving in game-based learning. Journal of Educational Technology & Society, 17(1), 42–53.
Facey-Shaw, L., Specht, M., van Rosmalen, P., & Bartley-Bryan, J. (2020). Do badges affect intrinsic motivation in introductory programming students? Simulation & Gaming, 51(1), 33–54.
Faiella, F., & Ricciardi, M. (2015). Gamification and learning: A review of issues and research. Journal of e-Learning and Knowledge Society, 11(3), 13–21.
Hedges, L. V. (1981). Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128.
Higgins, J. P., & Green, S. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions (Vol. 4). John Wiley & Sons.
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. British Medical Journal, 327(7414), 557–560.
Huang, R., Ritzhaupt, A. D., Sommer, M., Zhu, J., Stephen, A., Valle, N., et al. (2020). The impact of gamification in educational settings on student learning outcomes: A meta-analysis. Educational Technology Research and Development, 68(4), 1875–1901.
Hung, H. C., Young, S. S. C., & Lin, C. P. (2015a). No student left behind: A collaborative and competitive game-based learning environment to reduce the achievement gap of EFL students in Taiwan. Technology, Pedagogy and Education, 24(1), 35–49.
Hwang, G. H., Chen, B., Chen, R. S., Wu, T. T., & Lai, Y. L. (2019). Differences between students’ learning behaviors and performances of adopting a competitive game-based item bank practice approach for learning procedural and declarative knowledge. Interactive Learning Environments, 27(5–6), 740–753.
Immorlica, N., Stoddard, G., & Syrgkanis, V. (2015, May). Social status and badge design. In proceedings of the 24th international conference on world wide web (pp. 473-483). International world wide web conferences steering committee.
Johnson, D. W., Maruyama, G., Johnson, R., Nelson, D., & Skon, L. (1981). Effects of cooperative, competitive, and individualistic goal structures on achievement: A meta-analysis. Psychological Bulletin, 89(1), 47–62.
Karakostas, A., & Demetriadis, S. (2011). Enhancing collaborative learning through dynamic forms of support: The impact of an adaptive domain-specific support strategy. Journal of Computer Assisted Learning, 27(3), 243–258.
Katz, B., Jaeggi, S., Buschkuehl, M., Stegman, A., & Shah, P. (2014). Differential effect of motivational features on training improvements in school-based cognitive training. Frontiers in Human Neuroscience, 8(1), 242–251.
Kim, E., Rothrock, L., & Freivalds, A. (2018). An empirical study on the impact of lab gamification on engineering students' satisfaction and learning. International Journal of Engineering Education, 34(1), 201–216.
Lam, S. F., Yim, P. S., Law, J. S., & Cheung, R. W. (2004). The effects of competition on achievement motivation in Chinese classrooms. British Journal of Educational Psychology, 74(2), 281–296.
Landers, R. N., Bauer, K. N., Callan, R. C., & Armstrong, M. B. (2015). Psychological theory and the gamification of learning. In T. Reiners & L. Wood (Eds.), Gamification in education and business. Springer.
Lou, Y., Abrami, P. C., & d’Apollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71(3), 449–521.
Matsuda, N., Yarzebinski, E., Keiser, V., Raizada, R., Stylianides, G. J., & Koedinger, K. R. (2013). Studying the effect of a competitive game show in a learning by teaching environment. International Journal of Artificial Intelligence in Education, 23(1–4), 1–21.
Michaels, J. W. (1977). Classroom reward structures and academic performance. Review of Educational Research, 47(1), 87–98.
Mullins, D., Rummel, N., & Spada, H. (2011). Are two heads always better than one? Differential effects of collaboration on students’ computer-supported learning in mathematics. International Journal of Computer-Supported Collaborative Learning, 6(3), 421–443.
Pareto, L., Haake, M., Lindström, P., Sjödén, B., & Gulz, A. (2012). A teachable-agent-based game affording collaboration and competition: Evaluating math comprehension and motivation. Educational Technology Research and Development, 60(5), 723–751.
Plass, J. L., O'Keefe, P. A., Homer, B. D., Case, J., Hayward, E. O., Stein, M., & Perlin, K. (2013). The impact of individual, competitive, and collaborative mathematics game play on learning, performance, and motivation. Journal of Educational Psychology, 105(4), 1050–1066.
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231.
Reid, A. J., Paster, D., & Abramovich, S. (2015). Digital badges in undergraduate composition courses: Effects on intrinsic motivation. Journal of Computers in Education, 2(4), 377–398.
Rigby, S., & Ryan, R. M. (2011). Glued to games: How video games draw us in and hold us spellbound. Praeger.
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis. Publication bias in meta-analysis: Prevention, assessment and adjustments, 1–7.
Rubio-Aparicio, M., Sánchez-Meca, J., López-López, J. A., Botella, J., & Marín-Martínez, F. (2017). Analysis of categorical moderators in mixed-effects meta-analysis: Consequences of using pooled versus separate estimates of the residual between-studies variances. British Journal of Mathematical and Statistical Psychology, 70(3), 439–456.
Sailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior, 69, 371–380.
Sailer, M., & Homner, L. (2020). The gamification of learning: A Meta-analysis. Educational Psychology Review, 32, 77–112.
Scott, S., & Palincsar, A. (2013). Sociocultural theory. Education. http://dr-hatfield.com/theorists/resources/sociocultural_theory.pdf
Slavin, R. E. (1980). Cooperative learning. Review of Educational Research, 50(2), 315–342.
Skon, L., Johnson, D. W., & Johnson, R. T. (1981). Cooperative peer interaction versus individual competition and individualistic efforts: Effects on the acquisition of cognitive reasoning strategies. Journal of Educational Psychology, 73(1), 83–92.
Slavin, R. E. (1983). When does cooperative learning increase student achievement? Psychological Bulletin, 94(3), 429–445.
Slavin, R. E., Madden, N. A., & Leavey, M. (1984). Effects of team assisted individualization on the mathematics achievement of academically handicapped and nonhandicapped students. Journal of Educational Psychology, 76(5), 813–819.
Storch, N. (2011). Collaborative writing in L2 contexts: Processes, outcomes, and future directions. Annual Review of Applied Linguistics, 31, 275–288.
Thiebes, S., Lins, S., & Basten, D. (2014). Gamifying information systems-a synthesis of gamification mechanics and dynamics. In proceedings of the 20th European conference on information system, Tel Aviv, Israel.
Verenikina, I., Harris, P., & Lysaght, P. (2003, July). Child's play: Computer games, theories of play and children's development. In proceedings of the International Federation for Information Processing Working Group 3.5 open conference on Young children and learning technologies-Vol. 34 (pp. 99-106). Australian computer society, Inc.
Virvou, M., & Katsionis, G. (2008). On the usability and likeability of virtual reality games for education: The case of VR-ENGAGE. Computers & Education, 50(1), 154–178.
Vygotsky, L. S. (1977). The development of higher psychological functions. Soviet Psychology, 15(3), 60–73.
Wells GA, Shea B, O'Connell D, Peterson J, Welch V, Losos M, et al.(2014) The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses.
Werbach, K., & Hunter, D. (2012). For the win: How game thinking can revolutionize your business. Wharton Digital Press.
Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265.
Zainuddin, Z. (2018a). Students' learning performance and perceived motivation in gamified flipped-class instruction. Computers & Education, 126, 75–88.
Zainuddin, Z., Chu, S. K. W., Shujahat, M., & Perera, C. J. (2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30, 100326.
Studies Included in Meta-analysis
Abad, E., Gil, J., & Suárez, P. (2017). A game-based educational method relying on student-generated questions. The International Journal of Engineering Education, 33(6), 1786–1797.
Allen, L. K., Crossley, S. A., Snow, E. L., & McNamara, D. S. (2014). L2 writing practice: Game enjoyment as a key to engagement. Language Learning & Technology, 18(2), 124–150.
Bawa, P. (2019). Using Kahoot to inspire. Journal of Educational Technology Systems, 47(3), 373–390.
Buckley, P., & Doyle, E. (2016). Gamification and student motivation. Interactive Learning Environments, 24(6), 1162–1175.
Daubenfeld, T., & Zenker, D. (2014). A game-based approach to an entire physical chemistry course. Journal of Chemical Education, 92(2), 269–277.
De-Marcos, L., Domínguez, A., Saenz-de-Navarrete, J., & Pagés, C. (2014). An empirical study comparing gamification and social networking on e-learning. Computers & Education, 75, 82–91.
De-Marcos, L., Garcia-Cabot, A., & Garcia-Lopez, E. (2017). Towards the social gamification of e-learning: A practical experiment. International Journal of Engineering Education, 33(1), 66–73.
Eseryel, D., Law, V., Ifenthaler, D., Ge, X., & Miller, R. (2014b). An investigation of the interrelationships between motivation, engagement, and complex problem solving in game-based learning. Journal of Educational Technology & Society, 17(1), 42–53.
Gauthier, A., Corrin, M., & Jenkinson, J. (2015). Exploring the influence of game design on learning and voluntary use in an online vascular anatomy study aid. Computers & Education, 87, 24–34.
Göksün, D. O., & Gürsoy, G. (2019). Comparing success and engagement in gamified learning experiences via Kahoot and Quizizz. Computers & Education, 135, 15–29.
Hsu, C. C., & Wang, T. I. (2018). Applying game mechanics and student-generated questions to an online puzzle-based game learning system to promote algorithmic thinking skills. Computers & Education, 121, 73–88.
Hulse, T., Daigle, M., Manzo, D., Braith, L., Harrison, A., & Ottmar, E. (2019). From here to there! Elementary: A game-based approach to developing number sense and early algebraic understanding. Educational Technology Research and Development, 67(2), 423–441.
Hung, H. C., Young, S. S. C., & Lin, C. P. (2015b). No student left behind: A collaborative and competitive game-based learning environment to reduce the achievement gap of EFL students in Taiwan. Technology, Pedagogy and Education, 24(1), 35–49.
Jo, J., Jun, H., & Lim, H. (2018). A comparative study on gamification of the flipped classroom in engineering education to enhance the effects of learning. Computer Applications in Engineering Education, 26(5), 1626–1640.
Liao, C. C., Chang, W. C., & Chan, T. W. (2018). The effects of participation, performance, and interest in a game-based writing environment. Journal of Computer Assisted Learning, 34(3), 211–222.
Lin, K. C., Wei, Y. C., & Hung, J. C. (2012). The effects of online interactive games on high school students’ achievement and motivation in history learning. International Journal of Distance Education Technologies (IJDET), 10(4), 96–105.
Lu, S. J., & Liu, Y. C. (2015). Integrating augmented reality technology to enhance children’s learning in marine education. Environmental Education Research, 21(4), 525–541.
Milner, R., Parrish, J., Wright, A., Gnarpe, J., & Keenan, L. (2015). Research and teaching: Exploring the use of an online Quiz game to provide formative feedback in a large-enrollment, introductory biochemistry course. Journal of College Science Teaching, 45(2), 86–97.
Neef, N. A., Perrin, C. J., Haberlin, A. T., & Rodrigues, L. C. (2011). Studying as fun and games: effects on college students'quiz performance. Journal of Applied Behavior Analysis, 44(4), 897–901.
Ortiz-Rojas, M., Chiluiza, K., & Valcke, M. (2019). Gamification through leaderboards: An empirical study in engineering education. Computer Applications in Engineering Education, 27(4), 777–788.
Panoutsopoulos, H., & Sampson, D. (2012). A study on exploiting commercial digital games into school context. Educational Technology and Society, 15(1), 15–27.
Pareto, L. (2014). A teachable agent game engaging primary school children to learn arithmetic concepts and reasoning. International Journal of Artificial Intelligence in Education, 24(3), 251–283.
Tsay, C. H. H., Kofinas, A., & Luo, J. (2018). Enhancing student learning experience with technology-mediated gamification: An empirical study. Computers & Education, 121, 1–17.
Tsai, F. H. (2017). An investigation of gender differences in a game-based learning environment with different game modes. Eurasia Journal of Mathematics Science and Technology Education, 13(7), 3209–3226.
Turan, Z., & Meral, E. (2018). Game-based versus to non-game-based: The impact of student response systems on Students' achievements, engagements and test anxieties. Informatics in Education, 17(1), 105–116.
van den Heuvel-Panhuizen, M., Kolovou, A., & Robitzsch, A. (2013). Primary school students’ strategies in early algebra problem solving supported by an online game. Educational Studies in Mathematics, 84(3), 281–307.
Yen, L., Chen, C. M., & Huang, H. B. (2016, June). Effects of mobile game-based English vocabulary learning APP on learners' perceptions and learning performance: A case study of Taiwanese EFL learners. In international conference on e-learning (p. 255). Academic conferences international limited.
Young, S. S. C., & Wang, Y. H. (2014). The game embedded CALL system to facilitate English vocabulary acquisition and pronunciation. Journal of Educational Technology & Society, 17(3), 239–251.
Zainuddin, Z. (2018b). Students' learning performance and perceived motivation in gamified flipped-class instruction. Computers & Education, 126, 75–88.
Acknowledgments
The authors would like to thank Prof. Catherine McBride, Prof. Wai Chan, Prof. Suzanne So and Prof. Scott Aubrey for their feedback on this manuscript.
Availability of data and material
The coding sheet will be made available to the public as supplementary documents in order to achieve data transparency.
Funding
No funding was received to assist with the preparation of this manuscript.
Author information
Authors and Affiliations
Contributions
Jana Chi-San Ho and Letty Y.-Y. Kwan contributed to the study conception and design. The search of included primary studies, coding process and data analysis were performed by Jana Chi-San Ho. Yu-Sheng Hung was the second coder and double-checked data analysis. The first draft of the manuscript was written by Jana Chi-San Ho and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflicts of interest/competing interests
The author(s) declare(s) that they have no competing interests.
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Code availability
Not applicable.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1
See Table 3.
Appendix 2
1.1 Risk of Bias
Note: A study can be awarded a maximum of one star for each numbered item within the Selection and Outcome categories.
Selection
-
1)
Representativeness of the gaming condition
-
a)
truly representative of the average students in educational settings*
-
b)
somewhat representative of the average students in educational settings*
-
c)
selected group of users
-
d)
no description of the derivation of the condition
-
2)
Selection of the non-gaming condition
-
a)
drawn from the same community as the gaming condition *
-
b)
drawn from a different source
-
c)
no description of the derivation of the non-gaming condition
-
3)
Ascertainment of gamification
-
a)
features of gamification were demonstrated *
-
b)
game designs were illustrated *
-
c)
no description
-
4)
Demonstration that learning performance was equal at start of study
-
a)
yes *
-
b)
no
Comparability.
Comparability of conditions on the basis of the design or analysis
-
1)
study controls for pre-condition scores for learning performance *
-
2)
participation in the gaming condition does not directly affect learning performance (e.g., no extra points are rewarded toward the test score for participation in games)*
Outcome
-
1)
Assessment of learning performance
-
a)
record linkage (e.g., test score) *
-
b)
knowledge or skills measured *
-
c)
others
-
2)
Was follow-up long enough (> two weeks) for the effect of gamification to occur
-
a)
yes *
-
b)
no
-
c)
unclear
-
3)
Adequacy of follow up of conditions
-
a)
complete follow up - all subjects accounted for *
-
b)
subjects lost to follow up unlikely to introduce bias - small number lost - > 90% follow up, or description provided of those lost) *
-
c)
follow up rate < 90% and no description of those lost
-
d)
no statement
Appendix 3
See Table 5.
Rights and permissions
About this article
Cite this article
Ho, J.CS., Hung, YS. & Kwan, L.YY. The impact of peer competition and collaboration on gamified learning performance in educational settings: a Meta-analytical study. Educ Inf Technol 27, 3833–3866 (2022). https://doi.org/10.1007/s10639-021-10770-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-021-10770-2