Abstract
Computational thinking is considered to be an important competence in the intelligent era, and the incorporation of computational thinking as an integral part of school education beginning in childhood has been proposed. However, the ways in which computational thinking can be taught more effectively the context of in K-12 programming teaching remain unclear. This paper reports the results of a meta-analysis of 28 empirical studies on K-12 programming teaching that were published in international education journals in the 21st century to determine which teaching methods and programming tools are most effective in promoting the computational thinking of K-12 students. The results show that (1) programming teaching can promote the improvement of K-12 students’ computational thinking (ES = 0.72, z = 9.9, P < 0.01), with an overall effect at the upper-middle level (95% CI[0.60,0.83]); (2) scaffolding programming (ES = 1.84, z = 11.9, P < 0.01) and problem-based programming (ES = 1.14, z = 5.57, P < 0.01) are the most effective teaching methods and can significantly promote the development of K-12 students’ computational thinking (chi2 = 40.58, P < 0.01); (3) since differences in the effect of programming tools between groups are not significant (Chi2 = 6.47, P = 0.09), it is impossible to determine which programming tools are most effective; and (4) intervention duration (ES = 0.72, z = 11.9, P < 0.05, 95% CI[0.60, 0.83]) and learning scaffold (ES = 0.83, z = 6.27, P < 0.05, 95% CI[0.57, 1.09]) are both key moderating variables that affect the improvement of computational thinking. Based on these results, suggestions are provided for future research and practice.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets supporting the conclusions of this article are included within the article or contact the first author of this article to obtain the original data.
References
Angeli, C., & Giannakos, M. (2020). Computational thinking education: issues and challenges. Computers in Human Behavior, 105, 106-185. https://doi.org/10.1016/j.chb.2019.106185.
Beck, L., & Chizhik, A. (2013). Cooperative Learning Instructional Methods for CS1:Design, Implementation and Evaluation. ACM Transactions on Computing Education, 13(3), 1–21. https://doi.org/10.1145/2499947.2492686.
Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. In 2012 annual meeting of the American educational research association (pp. 1–25).
Bulu, S. T., & Pedersen, S. (2012). Supporting problem-solving performance in a hypermedia learning environment:the role of students’ prior knowledge and meta-cognitive skills. Computers in Human Behavior, 28(4), 1162–1169. https://doi.org/10.1016/j.chb.2012.01.026.
Chen, J., Wang, M., Kischner, P., & Tsai, C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584.
Chen, C. H., Liu, T. K., & Huang, K. (2021). Scaffolding vocational high school students’ computational thinking with cognitive and metacognitive prompts in learning about programmable logic controllers. Journal of Research on Technology in Education, 20, 1–18. https://doi.org/10.1080/15391523.2021.1983894.
Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21–29. https://doi.org/10.2307/30218684.
Cohen, A. (1992). Antecedents of organizational commitment across occupational groups: a meta-analysis. Journal of Organizational Behavior. https://doi.org/10.1002/job.4030130602.
Cooper, H. (2010). Research synthesis and meta-analysis:A step-by-step approach (4th ed.). Sage.
Denner, J., Werner, L., Campe, S., & Ortiz, E. (2014). Pair programming: under what conditions is it advantageous for middle school students? Journal of Research on Technology in Education, 46(3), 277–296. https://doi.org/10.1080/15391523.2014.888272.
Denner, J., Werner, L., & Ortiz, E. (2012). Computer games created by middle school girls: can they be used to measure understanding of computer science concepts? Computers & Education, 58(1), 240–249. https://doi.org/10.1016/j.compedu.2011.08.006.
Denny, P., Cukierman, D., & Bhaskar, J. (2015). Measuring the effect of inventing practice exercises on learning in an introductory programming course. Koli Calling Conference on Computing Education Research. ACM.
Florez, F. B., Casallas, R., Hernandez, M., Reyes, A., Restrepo, S., & Danies, G. (2017). Changing a generation’ s way of thinking: Teaching computational thinking through programming. Review of Educational Research, 87(4), 834–860. https://doi.org/10.3102/0034654317710096.
Grover, S., & Pea, R. (2013). Computational thinking in k-12: a review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051.
Halpern. (2001). Assessing the effectiveness of critical thinking instruction. The Journal of General Education, 50(4), 270–286. https://doi.org/10.2307/27797889.
Hsu, T., Chang, S., & Hung, Y. (2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers Education, 126, 296–310. https://doi.org/10.1016/j.compedu.2018.07.004.
Jun, S. J., Han, S. K., & Kim, S. H. (2016). Effect of design-based learning on improving computational thinking. Behaviour & Information Technology, 1–11. https://doi.org/10.1080/0144929x.2016.1188415.
Kalelioglu, F., & Gulbahar, Z. (2014). The effects of teaching programming via scratch on problem solving skills: A discussion from learners’ perspective. Informatics in Education, 13(1), 33–50.
Kim, Y. (2020). The effects of pbl-based data science education classes using app inventor on elementary student’ computational thinking and creativity improvement. Journal of the Korean Association of Information Education, 24(6), 551–562. https://doi.org/10.14352/jkaie.2020.24.6.551.
Lai, X., & Wong, K. W. (2022). Collaborative versus individual problem solving in computational thinking through programming: a meta-analysis. British Journal of Educational Technology, 53, 150–170. https://doi.org/10.1111/bjet.13157.
Lee, Y. Y., Mauriello, M. L., Ahn, J., & Bederson, B. B. (2014). CTArcade: Computational thinking with games in school age children. International Journal of Child-Computer Interaction, 2, 26–33. https://doi.org/10.1016/j.ijcci.2014.06.003.
Lei, H., Chiu, M. M., Li, F., Wang, X., & Geng, Y. J. (2020). Computational thinking and academic achievement: a meta-analysis among students. Children and Youth Services Review, 118105439. https://doi.org/10.1016/j.childyouth.2020.105439.
Lipsey, M., & Wilson, D. (2000). Practical meta-analysis. Internatinal Educational and Professional, 92–160. London
Lieto, M., Inguaggiato, E., Castro, E., Cecchi, F., & Dario, P. (2017). Educational robotics intervention on executive functions in preschool children: a pilot study. Computers in Human Behavior, 71(C), 16–23. https://doi.org/10.1016/j.chb.2017.01.018.
Lye, S. Y., & Koh, J. (2014). Review on teaching and learning of computational thinking through programming: what is next for k-12? Computers in Human Behavior, 41(dec.), 51–61. https://doi.org/10.1016/j.chb.2014.09.012.
Lu, M., Odell, K., & Epp, C. (2022). A scoping review of computational thinking assessments in higher education. Journal of Computing in Higher Education, 34, 416–461. https://doi.org/10.1007/s12528-021-09305-y.
Morris. (2008). Estimating effect size from the pretest-posttest-control design. Organizational Research Methods, 11, 364–386.
Ministry of Education of the People’s Republic of China. (2022). Information technology curriculum standard for compulsory education [2022-05-07]. http://www.moe.gov.cn/srcsite/A26/s8001/202204/W020220420582361024968.pdf.
Ministry of Education of the People's Republic of China. (2020). Information technology curriculum standard for senior high School [2020-05-20]. http://www.moe.gov.cn/srcsite/A26/s8001/202006/t20200603_462199.html.
Papadakis, S. (2018). Is pair programming more effective than solo programming for secondary education novice programmers? A case study. International Journal of Web-Based Learning and Teaching Technologies, 13(1), 1–16. https://doi.org/10.4018/ijwltt.2018010101.
Pérez-Marín, R., Hijón-Neira, B. A., & Pizarro, C. (2018). Can computational thinking be improved by using a methodology based on metaphors and scratch to teach computer programming to children? Computers in Human Behavior, 105, 105849. https://doi.org/10.1016/j.chb.2018.12.027.
Reiser, & Brian, J. (2004). Scaffolding complex learning: The mechanisms of structuring and problematizing student work. Journal of the Learning Sciences, 13(3), 273–304. https://doi.org/10.2307/1466939.
Repenning, A., Webb, D., & Ioannidou, A. (2010). Scalable game design and the development of a checklist for getting computational thinking into public schools. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education (SIGCSE’10), 265–269. ACM Press.
Rn, E., & Rn, M. (2002). Critical thinking in nursing education: Literature review. Mary Courtney, 8(2), 89–98.
Rogerson, C., & Scott, E. (2010). The fear factor: How it affects students learning to program in a tertiary environment. Journal of Information Technology Education, 9(1), 147–171. https://doi.org/10.28945/1183.
Roman Gonzalez, M., Moreno-Leon, J., & Robles, G. (2019). Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In S. Kong, & H. Abelson (Eds.), Computational thinking education (pp. 79–98). Springer. https://doi.org/10.1007/978-981-13-6528-7_6.
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias and meta-analysis: prevention, assessments and adjustments, 261–286. https://doi.org/10.1002/0470870168.ch14.
Saad, A., & Zainudin, S. (2022). A review of project-based learning (pbl) and computational thinking (ct) in teaching and learning. Learning and Motivation, 78(12). https://doi.org/10.1016/j.lmot.2022.101802.
Scherer, R. (2016). Learning from the past-the need for empirical evidence on the transfer effects of computer programming skills. Frontiers in Psychology, 7(1390). https://doi.org/10.3389/fpsyg.2016.01390.
Sengupta, D., Farris, V., & Wright, M. (2015). Programming in K-12 science classrooms. Communications of the ACM, 58(11), 34–35. https://doi.org/10.1145/2822517.
Selby, C., & Woollard, J. (2013). Computational thinking: The developing definition[DB/OL]. [2021-07-12]. http://eprints.soton.ac.uk/356481.
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003.
Siddiq, F., & Scherer, R. (2017). Revealing the processes of students’interaction with a novel collaborative problem solving task: An in-depth analysis of think-aloud protocols. Computers in Human Behavior, 76, 509–525. https://doi.org/10.1016/j.chb.2017.08.007.
Sison, R. (2008). Investigating pair programming in a software engineering course in an Asian setting. In 2008 15th Asia-Pacific Software Engineering Conference (pp. 325–331). IEEE.
Sun, L., Guo, Z., & Hu, L. (2021). Educational games promote the development of students’ computational thinking: a meta-analytic review. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1931891.
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798. https://doi.org/10.1016/j.compedu.2019.103798.
Tikva, C., & Tambouris, E. (2021). Mapping computational thinking through programming in k-12 education: a conceptual model based on a systematic literature review. Computers & Education, 162(1), 104083. https://doi.org/10.1016/j.compedu.2020.104083.
Vahldicka, A., Faraha, P. R., Marcelino, M. J., & Mendes, A. J. (2020). A blocks-based serious game to support introductory computer programming in undergraduate education. Computers in Human Behavior Reports, 2, 100037. https://doi.org/10.1016/j.chbr.2020.100037.
Wang, C., Shen, J., & Chao, J. (2021). Integrating computational thinking in STEM education: A literature review. International Journal of Science and Mathematics Education. https://doi.org/10.1007/s10763-021-10227-5.
Wei, X., Lin, L., Meng, N., Tan, W., Kong, S. C., & Kinshuk. (2021). The effectiveness of partial pair programming on elementary school students’ computational thinking skills and self-efficacy. Computers & Education, 160, 104023. https://doi.org/10.1016/j.compedu.2020.104023.
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35.
Wing, J. M. (2010). Computational thinking: What and why? The link magazine [2022-05-20]. Retrieved fromhttp://www.cs.cmu.edu/~CompThink/resources/TheLinkWing.pdf.
Witherspoon, E. B., Higashi, R. M., Schunn, C. D., Baehr, E. C., & Shoop, R. (2017). Developing computational thinking through a virtual robotics programming curriculum. ACM Transactions on Computing Education, 18(1), 1–20.
Wood, D., Bruner, J. S., & Ross, G. (2006). The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x.
Yuksel, H. (2015). The effect of the computer assisted instruction on the academic achievement and retention of technical programme students’ in vocational foreign language. Procedia-Social and Behavioral Sciences, 174, 2513–2518. https://doi.org/10.1016/j.sbspro.2015.01.924.
Zhang, J.B., & Ji, L.Y. (2018). Evolution of the orientation of information technology education in primary and secondary schools. Research on Audio Visual Education, 39(05), 108–114. https://doi.org/10.13811/j.cnki.eer.2018.05.016.
Zhang, Y., Luo, R., Zhu, Y., & Yin, Y. (2021). Educational robots improve k-12 students’ computational thinking and stem attitudes: systematic review. Journal of Educational Computing Research, 59(7), 1450–1481. https://doi.org/10.1177/0735633121994070.
Funding
This work was supported by the graduate scientific research and innovation project of Xinjiang Uygur Autonomous Region named “Research on In-Depth Learning of High School Information Technology Courses for the Cultivation of Computing Thinking” (Project No.: XJ2022G190), and the independent innovation fund project for doctoral students of the College of Educational Science of Xinjiang Normal University named “Research on Project-based Teaching of High School Information Technology Courses From the Perspective of Discipline Core Literacy” (Project No.: XJNUJKYA2003).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Enwei Xu and Qingxia Wang conducted the literature search and were involved in the analysis and interpretation of data as well as writing-review and editing. Enwei Xu and Wei Wang drafted and critically revised the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of competing interest
All authors have no relevant financial or non-financial competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, E., Wang, W. & Wang, Q. A meta-analysis of the effectiveness of programming teaching in promoting K-12 students’ computational thinking. Educ Inf Technol 28, 6619–6644 (2023). https://doi.org/10.1007/s10639-022-11445-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-022-11445-2