Skip to main content
SpringerLink
Log in

The Evolution of Computing Education Research: A Meta-Analytic Perspective

  • Chapter
  • First Online:
Past, Present and Future of Computing Education Research

  • 504 Accesses

Abstract

Computing education research (CER) has gradually built its own identity as a research field. A part of this process has been the publication of a growing number of meta-studies which have explored the CER literature to identify the state of the art and trends in terms of active research topics, the nature of publications, use of different research methods, and the use and development of theoretical frameworks in papers published in different conferences and journals. In this chapter, we explore these meta-studies to build a picture of the way in which research in the field has developed. We identify trends over years, and discuss the implications of these findings for the future of the field.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    While instruments are generally considered methodological tools to measure something, we take here the perspective that they are theory-informed constructs which support the implementation of some specific theory or theories in research.

References

  1. Ahadi, A., Hellas, A., Ihantola, P., Korhonen, A., Petersen, A.: Replication in computing education research: researcher attitudes and experiences. In: Proceedings of the 16th Koli calling international conference on computing education research, pp. 2–11 (2016)

    Google Scholar 

  2. Alaqsam, A., Ghabban, F., Ameerbakhsh, O., Alfadli, I., Fayez, A.: Current trends in online programming languages learning tools: a systematic literature review. Journal of Software Engineering and Applications 14(7), 277–297 (2021)

    Article  Google Scholar 

  3. Atchison, W.F., Conte, S.D., Hamblen, J.W., Hull, T.E., Keenan, T.A., Kehl, W.B., McCluskey, E.J., Navarro, S.O., Rheinboldt, W.C., Schweppe, E.J., Viavant, W., Young, D.M.: Curriculum 68: recommendations for academic programs in computer science: a report of the ACM curriculum committee on computer science. Communications of the ACM 11(3), 151–197 (1968)

    Article  Google Scholar 

  4. Austing, R.H., Barnes, B.H., Bonnette, D.T., Engel, G.L., Stokes, G.: Curriculum recommendations for the undergraduate program in computer science: a working report of the ACM committee on curriculum in computer sciences. ACM SIGCSE Bulletin 9(2), 1–16 (1977)

    Article  Google Scholar 

  5. Bandura, A.: Self-efficacy mechanism in human agency. American Psychologist 37(2), 122 (1982)

    Article  Google Scholar 

  6. Becker, B.A., Quille, K.: 50 years of CS1 at SIGCSE: a review of the evolution of introductory programming education research. In: 50th Technical Symposium on Computer Science Education, pp. 338–344 (2019)

    Google Scholar 

  7. Ben-Ari, M., Bednarik, R., Levy, R.B.B., Ebel, G., Moreno, A., Myller, N., Sutinen, E.: A decade of research and development on program animation: the Jeliot experience. Journal of Visual Languages & Computing 22(5), 375–384 (2011)

    Article  Google Scholar 

  8. Berglund, A., Box, I., Eckerdal, A., Lister, R., Pears, A.: Learning educational research methods through collaborative research: the PhICER initiative. In: Tenth Australasian Computing Education Conference, pp. 35–42 (2008)

    Google Scholar 

  9. Boustedt, J.: Students’ understanding of the concept of interface in a situated context. Computer Science Education 19(1), 15–36 (2009)

    Article  Google Scholar 

  10. Brown, N.C., Altadmri, A., Sentance, S., Kölling, M.: Blackbox, five years on: an evaluation of a large-scale programming data collection project. In: 14th International Computing Education Research Conference, pp. 196–204 (2018)

    Google Scholar 

  11. Chahal, K.K., Kaur, A., Saini, M.: Empirical studies on using pair programming as a pedagogical tool in higher education courses: a systematic literature review. Research and Evidence in Software Engineering, pp. 251–286 (2021)

    Google Scholar 

  12. Decker, A., McGill, M.M.: A topical review of evaluation instruments for computing education. In: 50th Technical Symposium on Computer Science Education, pp. 558–564 (2019)

    Google Scholar 

  13. Denning, P.J., Tedre, M.: Computational Thinking. MIT Press (2019)

    Google Scholar 

  14. Dorn, B., Elliott Tew, A.: Empirical validation and application of the computing attitudes survey. Computer Science Education 25(1), 1–36 (2015)

    Article  Google Scholar 

  15. Du, Y., Luxton-Reilly, A., Denny, P.: A review of research on parsons problems. In: Proceedings of the Twenty-Second Australasian Computing Education Conference, pp. 195–202 (2020)

    Google Scholar 

  16. Dweck, C.S.: Self-theories: their role in motivation, personality, and development. Psychology Press (2013)

    Google Scholar 

  17. Fensham, P.J.: Defining an Identity: The Evolution of Science Education as a Field of Research. Springer Science & Business Media (2004)

    Google Scholar 

  18. Fincher, S., Petre, M.: Computer Science Education Research. CRC Press (2004)

    Google Scholar 

  19. Fincher, S.A., Robins, A.V.: The Cambridge Handbook of Computing Education Research. Cambridge University Press (2019)

    Google Scholar 

  20. Gregor, S.: The nature of theory in information systems. MIS Quarterly, pp. 611–642 (2006)

    Google Scholar 

  21. Guzdial, M.: Exploring hypotheses about media computation. In: Ninth International Computing Education Research Conference, pp. 19–26 (2013)

    Google Scholar 

  22. Guzdial, M., du Boulay, B.: The history of computing. The Cambridge Handbook of Computing Education Research (2019) 11 (2019)

    Google Scholar 

  23. Heckman, S., Carver, J.C., Sherriff, M., Al-Zubidy, A.: A systematic literature review of empiricism and norms of reporting in computing education research literature. ACM Transactions on Computing Education 22(1), 1–46 (2021)

    Article  Google Scholar 

  24. Hundhausen, C.D., Douglas, S.A., Stasko, J.T.: A meta-study of algorithm visualization effectiveness. Journal of Visual Languages & Computing 13(3), 259–290 (2002)

    Article  Google Scholar 

  25. Kaplan, A., Maehr, M.L.: The contributions and prospects of goal orientation theory. Educational Psychology Review 19(2), 141–184 (2007)

    Article  Google Scholar 

  26. Kelleher, C., Pausch, R.: Lowering the barriers to programming: a taxonomy of programming environments and languages for novice programmers. Computing Surveys (CSUR) 37(2), 83–137 (2005)

    Article  Google Scholar 

  27. Kinnunen, P., Meisalo, V., Malmi, L.: Have we missed something? Identifying missing types of research in computing education. In: Sixth International Computing Education Research Workshop, pp. 13–22 (2010)

    Google Scholar 

  28. Kinnunen, P., Simon, B.: My program is ok – am I? Computing freshmen’s experiences of doing programming assignments. Computer Science Education 22(1), 1–28 (2012)

    Google Scholar 

  29. Kölling, M., Quig, B., Patterson, A., Rosenberg, J.: The BlueJ system and its pedagogy. Computer Science Education 13(4), 249–268 (2003)

    Article  Google Scholar 

  30. Kong, S.C., Chiu, M.M., Lai, M.: A study of primary school students’ interest, collaboration attitude, and programming empowerment in computational thinking education. Computers & Education 127, 178–189 (2018)

    Article  Google Scholar 

  31. Lewis, C.M.: Exploring variation in students’ correct traces of linear recursion. In: Tenth International Computing Education Research Conference, pp. 67–74 (2014)

    Google Scholar 

  32. Lishinski, A., Good, J., Sands, P., Yadav, A.: Methodological rigor and theoretical foundations of CS education research. In: 12th International Computing Education Research Conference, pp. 161–169 (2016)

    Google Scholar 

  33. Lister, R.: The Randolph thesis: CSEd research at the crossroads. SIGCSE Bulletin 39(4), 16–18 (2007)

    Article  Google Scholar 

  34. Lopez, M., Whalley, J., Robbins, P., Lister, R.: Relationships between reading, tracing and writing skills in introductory programming. In: Proceedings of the Fourth International Workshop on Computing Education Research, pp. 101–112 (2008)

    Google Scholar 

  35. Lukkarinen, A., Malmi, L., Haaranen, L.: Event-driven programming in programming education: a mapping review. ACM Transactions on Computing Education 21(1), 1–31 (2021)

    Article  Google Scholar 

  36. Luxton-Reilly, A., Simon, Albluwi, I., Becker, B.A., Giannakos, M., Kumar, A.N., Ott, L., Paterson, J., Scott, M.J., Sheard, J., Szabo, C.: Introductory programming: a systematic literature review. In: ITiCSE 2018 Working Group Reports, pp. 55–106 (2018)

    Google Scholar 

  37. Malmi, L., Sheard, J., Kinnunen, P., Simon, Sinclair, J.: Computing education theories: what are they and how are they used? In: 15th International Computing Education Research Conference, pp. 187–197 (2019)

    Google Scholar 

  38. Malmi, L., Sheard, J., Kinnunen, P., Simon, Sinclair, J.: Theories and models of emotions, attitudes, and self-efficacy in the context of programming education. In: 16th International Computing Education Research Conference, p. 36–47 (2020)

    Google Scholar 

  39. Malmi, L., Sheard, J., Kinnunen, P., Simon, Sinclair, J.: Development and use of domain-specific learning theories, models and instruments in computing education. ACM Transactions on Computing Education, 23(1), Article 6, pp. 1–48 (2023)

    Google Scholar 

  40. Malmi, L., Sheard, J., Simon, Bednarik, R., Helminen, J., Kinnunen, P., Korhonen, A., Myller, N., Sorva, J., Taherkhani, A.: Theoretical underpinnings of computing education research: what is the evidence? In: Tenth International Computing Education Research Conference, pp. 27–34 (2014)

    Google Scholar 

  41. Malmi, L., Sheard, J., Simon, Bednarik, R., Helminen, J., Korhonen, A., Myller, N., Sorva, J., Taherkhani, A.: Characterizing research in computing education: a preliminary analysis of the literature. In: Sixth International Computing Education Research Workshop, pp. 3–12 (2010)

    Google Scholar 

  42. Margulieux, L., Ketenci, T.A., Decker, A.: Review of measurements used in computing education research and suggestions for increasing standardization. Computer Science Education 29(1), 49–78 (2019)

    Article  Google Scholar 

  43. McCauley, R., Grissom, S., Fitzgerald, S., Murphy, L.: Teaching and learning recursive programming: a review of the research literature. Computer Science Education 25(1), 37–66 (2015)

    Article  Google Scholar 

  44. McGill, M.M., Decker, A.: Construction of a taxonomy for tools, languages, and environments across computing education. In: 16th International Computing Education Research Conference, pp. 124–135 (2020)

    Google Scholar 

  45. McGill, M.M., Decker, A.: A gap analysis of statistical data reporting in K-12 computing education research: recommendations for improvement. In: 51st Technical Symposium on Computer Science Education, pp. 591–597 (2020)

    Google Scholar 

  46. McGill, M.M., Decker, A.: Tools, languages, and environments used in primary and secondary computing education. In: 25th Conference on Innovation and Technology in Computer Science Education, pp. 103–109 (2020)

    Google Scholar 

  47. Morrison, K., van der Werf, G.: Editorial. Educational Research and Evaluation 18(5), 399–401 (2012)

    Article  Google Scholar 

  48. Naps, T.L., Rößling, G., Almstrum, V., Dann, W., Fleischer, R., Hundhausen, C., Korhonen, A., Malmi, L., McNally, M., Rodger, S., Velásquez-Iturbide, J.Á.: Exploring the role of visualization and engagement in computer science education. In: ITiCSE 2002 Working Group Reports, pp. 131–152 (2002)

    Google Scholar 

  49. Nelson, G.L., Ko, A.J.: On use of theory in computing education research. In: 14th International Computing Education Research Conference, pp. 31–39 (2018)

    Google Scholar 

  50. Nikula, U., Gotel, O., Kasurinen, J.: A motivation guided holistic rehabilitation of the first programming course. ACM Transactions on Computing Education 11(4), 1–38 (2011)

    Article  Google Scholar 

  51. Papert, S.A.: Mindstorms: Children, Computers, and Powerful Ideas. Basic Books (2020)

    Google Scholar 

  52. Park, T.H., Saxena, A., Jagannath, S., Wiedenbeck, S., Forte, A.: Towards a taxonomy of errors in HTML and CSS. In: Ninth International Computing Education Research Conference, pp. 75–82 (2013)

    Google Scholar 

  53. Parsons, D., Haden, P.: Parson’s programming puzzles: a fun and effective learning tool for first programming courses. In: Eighth Australasian Computing Education Conference, pp. 157–163 (2006)

    Google Scholar 

  54. Peng, J., Yuan, B., Spector, J.M., Wang, M.: Integrating technology in programming learning and instruction: a critical review. International Journal of Smart Technology and Learning 1(4), 323–343 (2019)

    Article  Google Scholar 

  55. Petre, M., Sanders, K., McCartney, R., Ahmadzadeh, M., Connolly, C., Hamouda, S., Harrington, B., Lumbroso, J., Maguire, J., Malmi, L., McGill, M.M., Vahrenhold, J.: Mapping the landscape of peer review in computing education research. In: ITiCSE 2020 Working Group Reports, pp. 173–209 (2020)

    Google Scholar 

  56. Randolph, J., Bednarik, R., Silander, P., Gonzalez, J., Myller, N., Sutinen, E.: A critical analysis of the research methodologies reported in the full papers of the proceedings of ICALT 2004. In: Fifth International Conference on Advanced Learning Technologies, pp. 10–14 (2005)

    Google Scholar 

  57. Randolph, J.J.: Computer science education research at the crossroads: a methodological review of computer science education research, 2000–2005. Utah State University (2007)

    Google Scholar 

  58. Randolph, J.J.: A methodological review of the program evaluations in K-12 computer science education. Informatics in Education 7(2), 237–258 (2008)

    Article  Google Scholar 

  59. Randolph, J.J., Bednarik, R., Myller, N.: A methodological review of the articles published in the proceedings of Koli Calling 2001-2004. In: Fifth Finnish/Baltic Sea Conference on Computer Science Education, pp. 103–109 (2005)

    Google Scholar 

  60. Randolph, J.J., Julnes, G., Sutinen, E., Lehman, S.: A methodological review of computer science education research. Journal of Information Technology Education: Research 7(1), 135–162 (2008)

    Article  Google Scholar 

  61. Reeves, S., Albert, M., Kuper, A., Hodges, B.D.: Why use theories in qualitative research? British Medical Journal 337 (2008)

    Google Scholar 

  62. Research Council of Norway: The Role of Theory in Educational Research – Report from the March Seminar 2011 (2012)

    Google Scholar 

  63. Sanders, K., Sheard, J., Becker, B.A., Eckerdal, A., Hamouda, S., Simon: Inferential statistics in computing education research: a methodological review. In: 15th International Computing Education Research Conference, pp. 177–185 (2019)

    Google Scholar 

  64. Shaffer, C.A., Cooper, M.L., Alon, A.J.D., Akbar, M., Stewart, M., Ponce, S., Edwards, S.H.: Algorithm visualization: the state of the field. ACM Transactions on Computing Education 10(3), 1–22 (2010)

    Article  MATH  Google Scholar 

  65. Sim, T.Y., Lau, S.L.: Online tools to support novice programming: a systematic review. In: Second Conference on e-Learning, e-Management and e-Services (IC3e), pp. 91–96 (2018)

    Google Scholar 

  66. Simon: A classification of recent Australasian computing education publications. Computer Science Education 17(3), 155–169 (2007)

    Google Scholar 

  67. Simon: Koli Calling comes of age: an analysis. In: Seventh Baltic Sea Conference on Computing Education Research (Koli Calling 2007), pp. 119–126 (2008)

    Google Scholar 

  68. Simon: Informatics in Education and Koli Calling: a comparative analysis. Informatics in Education 8(1), 101–114 (2009)

    Google Scholar 

  69. Simon: Ten years of the Australasian Computing Education Conference. In: 11th Australasian Computing Education Conference, p. 157–164. AUS (2009)

    Google Scholar 

  70. Simon, Sheard, J.: Twenty-four years of ITiCSE papers. In: 25th Conference on Innovation and Technology in Computer Science Education, p. 5–11. Association for Computing Machinery, New York, NY, USA (2020)

    Google Scholar 

  71. Simon, S., Sheard, J., Carbone, A., De Raadt, M., Hamilton, M., Lister, R., Thompson, E.: Eight years of computing education papers at NACCQ. National Advisory Committee on Computing Qualifications (2008)

    Google Scholar 

  72. Soloway, E., Ehrlich, K.: Empirical studies of programming knowledge. IEEE Transactions on Software Engineering (5), 595–609 (1984)

    Article  Google Scholar 

  73. Sorva, J., Karavirta, V., Malmi, L.: A review of generic program visualization systems for introductory programming education. ACM Transactions on Computing Education 13(4), 1–64 (2013)

    Article  Google Scholar 

  74. Szabo, C., Falkner, N., Petersen, A., Bort, H., Cunningham, K., Donaldson, P., Hellas, A., Robinson, J., Sheard, J.: Review and use of learning theories within computer science education research: primer for researchers and practitioners. In: ITiCSE 2019 Working Group Reports, pp. 89–109 (2019)

    Google Scholar 

  75. Tedre, M., Simon, Malmi, L.: Changing aims of computing education: a historical survey. Computer Science Education 28(2), 158–186 (2018)

    Google Scholar 

  76. Tedre, M., Sutinen, E.: Three traditions of computing: what educators should know. Computer Science Education 18(3), 153–170 (2008)

    Article  Google Scholar 

  77. Thomas, L., Eckerdal, A., McCartney, R., Moström, J.E., Sanders, K., Zander, C.: Graduating students’ designs: through a phenomenographic lens. In: Tenth International Computing Education Research Conference, pp. 91–98 (2014)

    Google Scholar 

  78. Thompson, E., Kinshuk: The nature of an object-oriented program: how do practitioners understand the nature of what they are creating? Computer Science Education 21(3), 269–287 (2011)

    Article  Google Scholar 

  79. Thota, N., Berglund, A., Clear, T.: Illustration of paradigm pluralism in computing education research. In: 14th Australasian Computing Education Conference (2012)

    Google Scholar 

  80. Tsai, M.J., Wang, C.Y., Hsu, P.F.: Developing the computer programming self-efficacy scale for computer literacy education. Journal of Educational Computing Research 56(8), 1345–1360 (2019)

    Article  Google Scholar 

  81. Umapathy, K., Ritzhaupt, A.D.: A meta-analysis of pair-programming in computer programming courses: implications for educational practice. ACM Transactions on Computing Education 17(4), 1–13 (2017)

    Article  Google Scholar 

  82. Urquiza-Fuentes, J., Velázquez-Iturbide, J.A.: Pedagogical effectiveness of engagement levels–a survey of successful experiences. Electronic Notes in Theoretical Computer Science 224, 169–178 (2009)

    Article  MATH  Google Scholar 

  83. Valentine, D.W.: CS educational research: a meta-analysis of SIGCSE Technical Symposium proceedings. ACM SIGCSE Bulletin 36(1), 255–259 (2004)

    Article  Google Scholar 

  84. Vessey, I., Ramesh, V., Glass, R.L.: A unified classification system for research in the computing disciplines. Information and Software Technology 47(4), 245–255 (2005)

    Article  Google Scholar 

  85. Weinberg, G.M.: The Psychology of Computer Programming. Van Nostrand Reinhold New York (1971)

    Google Scholar 

  86. Wigfield, A., Eccles, J.S.: Expectancy-value theory of achievement motivation. Contemporary Educational Psychology 25(1), 68–81 (2000)

    Article  Google Scholar 

  87. Xie, B., Loksa, D., Nelson, G.L., Davidson, M.J., Dong, D., Kwik, H., Tan, A.H., Hwa, L., Li, M., Ko, A.J.: A theory of instruction for introductory programming skills. Computer Science Education 29(2-3), 205–253 (2019)

    Article  Google Scholar 

  88. Yuen, T.T., Robbins, K.A.: A qualitative study of students’ computational thinking skills in a data-driven computing class. ACM Transactions on Computing Education 14(4), 1–19 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Aalto University, Espoo, Finland

    Lauri Malmi

  2. University of Warwick, Coventry, UK

    Jane Sinclair

  3. Monash University, Melbourne, VIC, Australia

    Judy Sheard

  4. Wadalba, NSW, Australia

    Simon

  5. University of Helsinki, Helsinki, Finland

    Päivi Kinnunen

Authors
  1. Lauri Malmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jane Sinclair
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Judy Sheard
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Simon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Päivi Kinnunen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lauri Malmi .

Editor information

Editors and Affiliations

  1. University of Eastern Finland, Joensuu, Finland

    Mikko Apiola

  2. University of Eastern Finland, Joensuu, Finland

    Sonsoles López-Pernas

  3. University of Eastern Finland, Joensuu, Finland

    Mohammed Saqr

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Malmi, L., Sinclair, J., Sheard, J., Simon, Kinnunen, P. (2023). The Evolution of Computing Education Research: A Meta-Analytic Perspective. In: Apiola, M., López-Pernas, S., Saqr, M. (eds) Past, Present and Future of Computing Education Research . Springer, Cham. https://doi.org/10.1007/978-3-031-25336-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-25336-2_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-25335-5

  • Online ISBN: 978-3-031-25336-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation