Skip to main content

Advertisement

SpringerLink
Log in

Comparing badges and learning goals in low- and high-stakes learning contexts

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

Digital badges (i.e., digital credentials for achievements) have been suggested as a useful and scalable implementation of gamification. Digital badges (hereafter “badges”) provide two potential supports for learning: (1) badges provide support for motivation by rewarding achievement and (2) badges provide implicit learning goals. The present paper describes two experiments in which we investigated whether badges can support self-regulated learning by comparing their impact on learning with students given explicit goals for student learning, a key factor in self-regulated learning. Specifically, we compared the effects of badges and goal setting in a low-stakes learning context (Experiment 1; online extra credit unit) and a high-stakes learning context (Experiment 2; introductory Educational Psychology courses). In these two quasi-experiments, participants were randomly assigned to one of four groups: badge only, goal only, badge + goal, or control (i.e., no badge, no goal). Learning was measured by comparing performance on topics related to Turkish Culture (Experiment 1) or Educational Psychology (Experiment 2) at pre-test and post-test. Somewhat surprisingly, the results from both studies demonstrated no significant improvement in learning between groups. The discussion suggests that caution should be taken when incorporating badges in learning contexts and provides guidance on the conditions under which badges may be most effective for supporting learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1.  Analyses were conducted with these 15 participants intact. Results revealed that the goals group (M = 2.96) performed significantly better than the badges + goals group (M = 1.86) on the festivals module quiz, F(3,103) = 3.880, p = .01. Because this result does not change the overall findings of our study, we chose to adhere to our original exclusion criteria.

  2. We adapted items from the Achievement Goal Orientation questionnaire (Was 2006) to measure four motivational goal orientations: mastery, performance approach, performance avoidant, and work avoidant and the motivation section of the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al. 1991). Little variance in AGO scores was found across all groups, F(3,46) = 1.09, p = .36. Similarly, there was little variation in scores on the MSLQ subscales [control beliefs, F(3,47) = 1.18, p = .33; extrinsic motivation, F(3,47) > 1 p = .50; intrinsic motivation, F(3,47) > 1, p = .80; self-efficacy, F(3, 47) > 1, p = .94; task value, F(3,47) = 1.16, p = .21; test-anxiety, F(3,47) = 1.47, p = .24].

  3. The experiment was conducted in an introductory Educational Psychology course. This fifteen-week course surveys the field and covers one topic per week. Topics include cognitive and learning theories, developmental theories, theories of motivation, cognition & instruction, and classroom management. The course textbook (Eggen and Kauchak 2016) was the basis for lectures, discussions, and quizzes.

References

  • Abramovic, S., & Wardrip, P. (2016). Impact of badges on motivation to learn. In L. Y. Muilenburg & Z. L. Berge (Eds.), Digital badges in education: Trends, issues, and cases (pp. 53–61). Abingdon: Routledge.

    Google Scholar 

  • Abramovich, S., Schunn, C., & Higashi, R. M. (2013). Are badges useful in education? It depends upon the type of badge and expertise of learner. Educational Technology Research and Development, 61(2), 217–232. https://doi.org/10.1007/s11423-013-9289-2.

    Article  Google Scholar 

  • Ahn, J., Pellicone, A., & Butler, B. S. (2014). Open badges for education: What are the implications at the intersection of open systems and badging? Research in Learning Technology. https://doi.org/10.3402/rlt.v22.23563.

    Article  Google Scholar 

  • Antin, J., & Churchill, E. F. (2011). Badges in social media: A social psychological perspective. In Proceedings of the 2011 ACM CHI conference on human factors in computing systems, Vancouver, Canada.

  • Bjork, E. L., & Bjork, R. A. (2014). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher & J. Pomerantz (Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (2nd ed., pp. 59–68). New York: Worth.

    Google Scholar 

  • Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: Beliefs, techniques, and illusions. Annual Review of Psychology, 64, 417–444. https://doi.org/10.1146/annurev-psych-113011-143823.

    Article  Google Scholar 

  • Brophy, J. (2005). Goal theorists should move on from performance goals. Educational Psychologist, 40(3), 167–176. https://doi.org/10.1207/s15326985ep4003_3.

    Article  Google Scholar 

  • Cameron, J., Pierce, W. D., Banko, K. M., & Gear, A. (2005). Achievement-based rewards and intrinsic motivation: A test of cognitive mediators. Journal of Educational Psychology, 97(4), 641–655. https://doi.org/10.1037/0022-0663.97.4.641.

    Article  Google Scholar 

  • Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014). Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis. Psychological Bulletin, 140(4), 980. https://doi.org/10.1037/a0035661.

    Article  Google Scholar 

  • Cheng, Z., Watson, S. L., & Newby, T. J. (2018). Goal setting and open digital badges in higher education. TechTrends, 62(2), 190–196.

    Article  Google Scholar 

  • Cleary, T. J., & Zimmerman, B. J. (2001). Self-regulation differences during athletic practice by experts, non-experts, and novices. Journal of Applied Sport Psychology, 13(2), 185–206.

    Article  Google Scholar 

  • Cucchiara, S., Giglio, A., Persico, D., & Rafaghelli, J. E. (2014). Supporting self-regulated learning through digital badges: A case study. In Y. Cao, T. Väljataga, J. K. T. Tang, H. Leung, & M. Laanpere (Eds.), New horizons in web based learning. Paper presented at the international conference for web-based learning, Tallin, Estonia, 14–17 August (pp. 133–142). Basel: Springer. https://doi.org/10.1007/978-3-319-13296-9_15.

    Google Scholar 

  • Daumiller, M., & Dresel, M. (2018). Supporting self-regulated learning with digital media using motivational regulation and metacognitive prompts. The Journal of Experimental Education. https://doi.org/10.1080/00220973.2018.1448744.

    Article  Google Scholar 

  • Deci, E. L., & Ryan, R. M. (2012). Motivation, personality, and development within embedded social contexts: An overview of self-determination theory. In R. M. Ryan (Ed.), The Oxford handbook of human motivation (pp. 85–107). Oxford: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780195399820.013.0006.

    Chapter  Google Scholar 

  • Delello, J. A., Hawley, H., McWhorter, R. R., Gipson, C. S., & Deal, B. (2018). Gamifying education: Motivation and the implementation of digital badges for use in higher education. International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), 13(4), 17–33.

    Article  Google Scholar 

  • Derryberry, A., Everhart, D., & Knight, E. (2016). Badges and competencies: New currency for professional credentials. In L. Y. Muilenburg & Z. L. Berge (Eds.), Digital badges in education: Trends, issues, and cases (pp. 12–20). Abingdon: Routledge.

    Google Scholar 

  • Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Gamification in education: A systematic mapping study. Educational Technology and Society, 18(3), 75–88.

    Google Scholar 

  • Dickey, M. (2005). Engaging by design: How engagement strategies in popular computer and video games can inform instructional design. Educational Technology Research and Development, 53(2), 67–83. https://doi.org/10.1007/BF02504866.

    Article  Google Scholar 

  • Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention. Learning and Instruction, 22(4), 271–280. https://doi.org/10.1016/j.learninstruc.2011.08.003.

    Article  Google Scholar 

  • Eggen, P., & Kauchak, D. (2016). Educational psychology: Windows on classrooms. Upper Saddle River, NJ: Merrill Prentice Hall.

    Google Scholar 

  • Fajiculay, J. R., Parikh, B. T., Wright, C. V., & Sheehan, A. H. (2017). Student perceptions of digital badges in a drug information and literature evaluation course. Currents in Pharmacy Teaching and Learning, 9(5), 881–886.

    Article  Google Scholar 

  • Filsecker, M., & Hickey, D. T. (2014). A multilevel analysis of the effects of external rewards on elementary students’ motivation, engagement and learning in an educational game. Computers & Education, 75, 136–148. https://doi.org/10.1016/j.compedu.2014.02.008.

    Article  Google Scholar 

  • Gamrat, C., Zimmerman, H. T., Dudek, J., & Peck, K. (2014). Personalized workplace learning: An exploratory study on digital badging within a teacher professional development program. British Journal of Educational Technology, 45(6), 1136–1148.

    Article  Google Scholar 

  • Garnett, T., & Button, D. (2018). The use of digital badges by undergraduate nursing students: A three-year study. Nurse Education in Practice, 32, 1–8.

    Article  Google Scholar 

  • Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., & Knight, E. (2013). Digital badges in education. Education and Information Technologies, 20, 403–410. https://doi.org/10.1007/s10639-013-9291-7.

    Article  Google Scholar 

  • Grant, S. (2014). Badges: Show what you know. Young Adult Library Services, 12(2), 28–32.

    Google Scholar 

  • Haaranen, L., Ihantola, P., Hakulinen, L., & Korhonen, A. (2014). How (not) to introduce badges to online exercises. In Proceedings of the 45th ACM technical symposium on computer science education, Atlanta, USA (pp. 33–38). https://doi.org/10.1145/2538862.2538921.

  • Hakulinen, L., & Auvinen, T. (2014). The effect of gamification on students with different achievement goal orientations. In Proceedings of the 2014 international conference on teaching and learning in computing and engineering, Kuching, Malaysia (pp. 9–16). https://doi.org/10.1109/latice.2014.10.

  • Hamari, J. (2017). Do badges increase user activity? A field experiment on the effects of gamification. Computers in Human Behavior, 71, 469–478.

    Article  Google Scholar 

  • Hartwig, M. K., & Dunlosky, J. (2012). Study strategies of college students: Are self-testing and scheduling related to achievement? Psychonomic Bulletin Review, 19, 126–134. https://doi.org/10.3758/s13423-011-0181-y.

    Article  Google Scholar 

  • Hartwig, M. K., Was, C. A., Isaacson, R. A., & Dunlosky, J. (2012). General knowledge monitoring as a predictor of in-class exam performance. British Journal of Educational Psychology, 82, 456–468. https://doi.org/10.1111/j.2044-8279.2011.02038.x.

    Article  Google Scholar 

  • Hensiek, S., DeKorver, B. K., Harwood, C. J., Fish, J., O’Shea, K., & Towns, M. (2017). Digital badges in science: A novel approach to the assessment of student learning. Journal of College Science Teaching, 46(3), 28–33.

    Article  Google Scholar 

  • Hossain, Z., Bumbacher, E., Brauneis, A., Diaz, M., Saltarelli, A., Blikstein, P., et al. (2017). Design guidelines and empirical case study for scaling authentic inquiry-based science learning via open online courses and interactive biology cloud labs. International Journal of Artificial Intelligence in Education, 28, 478–507.

    Article  Google Scholar 

  • Ifenthaler, D., Bellin-Mularski, N., & Mah, D. K. (2016). Foundation of digital badges and micro-credentials. Cham: Springer.

    Book  Google Scholar 

  • Kitsantas, A., & Zimmerman, B. (2006). Enhancing self-regulation of practice: The influence of graphing and self-evaluative standards. Metacognition Learning, 1, 201–212. https://doi.org/10.1007/s11409-006-9000-7.

    Article  Google Scholar 

  • Kitsantas, A., & Zimmerman, B. J. (2002). Comparing self-regulatory processes among novice, non-expert, and expert volleyball players: A microanalytic study. Journal of Applied Sport Psychology, 14(2), 91–105.

    Article  Google Scholar 

  • Kyewski, E., & Krämer, N. C. (2018). To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course. Computers & Education, 118, 25–37.

    Article  Google Scholar 

  • Lane, C., & Yi, S. (2017). Playing with virtual blocks: Minecraft as a learning environment for practice and research. In F. C. Blumberg & P. J. Brooks (Eds.), Cognitive development in digital contexts (pp. 145–166). Amsterdam: Elsevier.

    Chapter  Google Scholar 

  • Lepper, M. R., & Greene, D. (Eds.). (2015). The hidden costs of reward: New perspectives on the psychology of human motivation. London: Psychology Press.

    Google Scholar 

  • Martí-Parreño, J., Galbis-Córdova, A., & Miquel-Romero, M. J. (2018). Students’ attitude towards the use of educational video games to develop competencies. Computers in Human Behavior, 81, 366–377.

    Article  Google Scholar 

  • McDaniel, R., Lindgren, R., & Friskics, J. (2012). Using badges for shaping interactions in online learning environments. In Proceedings of the 2012 IEEE international professional communication conference, Orlando, USA (pp. 1–4). https://doi.org/10.1109/ipcc.2012.6408619.

  • Moore-Russo, D., Wiss, A., & Grabowski, J. (2018). Integration of gamification into course design: A noble endeavor with potential pitfalls. College Teaching, 66(1), 3–5.

    Article  Google Scholar 

  • Morris, B., Croker, S., Zimmerman, C., Gill, D., & Romig, C. (2013). Gaming science: The “Gamification” of scientific thinking. Frontiers in Psychology, 4, 607. https://doi.org/10.3389/fpsyg.2013.00607.

    Article  Google Scholar 

  • Mozilla Foundation. (2011). Open badges for lifelong learning (White paper). Retrieved September 10, 2016 from https://wiki.mozilla.org/images/5/59/OpenBadges-Working-Paper_012312.pdf.

  • Muilenburg, L. Y., & Berge, Z. L. (Eds.). (2016). Digital badges in education: Trends, issues, and cases. Abingdon: Routledge.

    Google Scholar 

  • Newcombe, N. S., Ambady, N., Eccles, J., Gomez, L., Klahr, D., Linn, M., et al. (2009). Psychology’s role in mathematics and science education. American Psychologist, 64(6), 538–550.

    Article  Google Scholar 

  • Pierce, W. D., Cameron, J., Banko, K. M., & So, S. (2003). Positive effects of rewards and performance standards on intrinsic motivation. The Psychological Record, 53(4), 561–579. https://doi.org/10.1007/BF03395453.

    Article  Google Scholar 

  • Pintrich, P. R. (1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459–470. https://doi.org/10.1016/S0883-0355(99)00015-4.

    Article  Google Scholar 

  • Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). https://doi.org/10.13140/rg.2.1.2547.6968.

  • Randall, D. L., Harrison, J. B., & West, R. E. (2013). Giving credit where credit is due: Designing open badges for a technology integration course. TechTrends, 57(6), 88–95. https://doi.org/10.1007/s11528-013-0706-5.

    Article  Google Scholar 

  • Reid, A. J., Paster, D., & Abramovich, S. (2015). Digital badges in undergraduate composition courses: Effects on intrinsic motivation. Journal of Computers in Education, 2(4), 377–398.

    Article  Google Scholar 

  • Rughiniş, R., & Matei, S. (2013). Digital badges: Signposts and claims of achievement. In C. Stephanidis (Ed.), HCI international 2013—Posters’ extended abstracts. Communications in computer and information science (Vol. 374, pp. 84–88). Berlin: Springer. https://doi.org/10.1007/978-3-642-39476-8_18.

    Chapter  Google Scholar 

  • Ryan, R. M., & Deci, E. L. (2000). Intrinsic and extrinsic motivations: Classic definitions and new directions. Contemporary Educational Psychology, 25(1), 54–67. https://doi.org/10.1006/ceps.1999.1020.

    Article  Google Scholar 

  • Sailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior, 69, 371–380.

    Article  Google Scholar 

  • Sapiano, A. B., Sammut, R., & Trapani, J. (2018). The effectiveness of virtual simulation in improving student nurses’ knowledge and performance during patient deterioration: A pre and post test design. Nurse Education Today, 62, 128–133.

    Article  Google Scholar 

  • Schunk, D. H., & Zimmerman, B. J. (Eds.). (1998). Self-regulated learning: From teaching to self-reflective practice. New York: Guilford Press.

    Google Scholar 

  • Senko, C., Hulleman, C. S., & Harackiewicz, J. M. (2011). Achievement goal theory at the crossroads: Old controversies, current challenges, and new directions. Educational Psychologist, 46(1), 26–47. https://doi.org/10.1080/00461520.2011.538646.

    Article  Google Scholar 

  • Shields, R., & Chugh, R. (2017). Digital badges–rewards for learning? Education and Information Technologies, 22(4), 1817–1824.

    Article  Google Scholar 

  • Simões, J., Redondo, R. D., & Vilas, A. F. (2013). A social gamification framework for a K-6 learning platform. Computers in Human Behavior, 29(2), 345–353. https://doi.org/10.1016/j.chb.2012.06.007.

    Article  Google Scholar 

  • Sundre, D. L., & Kitsantas, A. (2004). An exploration of the psychology of the examinee: Can examinee self-regulation and test-taking motivation predict consequential and non-consequential test performance? Contemporary Educational Psychology, 29(1), 6–26. https://doi.org/10.1016/S0361-476X(02)00063-2.

    Article  Google Scholar 

  • Vlach, H. A., & Sandhofer, C. M. (2012). Distributing learning over time: The spacing effect in children’s acquisition and generalization of science concepts. Child Development, 83, 1137–1144. https://doi.org/10.1111/j.1467-8624.2012.01781.x.

    Article  Google Scholar 

  • Völlinger, V. A., Spörer, N., Lubbe, D., & Brunstein, J. C. (2017). A path analytic test of the reading strategies mediation model: Relating cognitive competences and motivational influences with individual differences in fifth-grade students’ reading comprehension. The Journal of Educational Research, 111, 733–745.

    Article  Google Scholar 

  • Warneken, F., & Tomasello, M. (2013). The emergence of contingent reciprocity in young children. Journal of Experimental Child Psychology, 116(2), 338–350. https://doi.org/10.1016/j.jecp.2013.06.002.

    Article  Google Scholar 

  • Was, C. (2006). Achievement goal orientation: Taking another look. Electronic Journal of Research in Educational Psychology, 4(3), 529–550.

    Google Scholar 

  • White, M., & Shellenbarger, T. (2018). Gamification of nursing education with digital badges. Nurse Educator, 43(2), 78–82.

    Article  Google Scholar 

  • Zimmerman, B. J., & Schunk, D. H. (Eds.). (2001). Self-regulated learning and academic achievements: Theoretical perspectives (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

Download references

Acknowledgements

The authors thank Chris Was for statistical consulting and Christine Murphy for creating the AGO and MSLQ Macros.

Author information

Authors and Affiliations

  1. Department of Educational Psychology, Kent State University, PO Box 5190, Kent, OH, 44242-0001, USA

    Bradley J. Morris, Colleen Dragovich, Rachael Todaro, Sebiha Balci & Eve Dalton

  2. Department of Lifespan Development and Educational Science, Kent State University, 405 White Hall, Kent, OH, 44242, USA

    Bradley J. Morris

Authors
  1. Bradley J. Morris
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Colleen Dragovich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachael Todaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sebiha Balci
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Eve Dalton
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bradley J. Morris.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morris, B.J., Dragovich, C., Todaro, R. et al. Comparing badges and learning goals in low- and high-stakes learning contexts. J Comput High Educ 31, 573–603 (2019). https://doi.org/10.1007/s12528-019-09228-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12528-019-09228-9

Keywords

Search

Navigation