skip to main content
10.1145/3421590.3421598acmotherconferencesArticle/Chapter ViewAbstractPublication PageswipsceConference Proceedingsconference-collections
research-article

Programming in primary education: towards a research based assessment framework

Published: 28 October 2020 Publication History

Abstract

In March 2017, the Swedish government decided to introduce digital competence - including programming - in primary school. As a consequence, every math and technology teacher in grades 1-9 in Sweden is expected to integrate programming in their teaching. Furthermore, the Swedish school law requires that teaching is based on scientific evidence and proven experience. In addition to professional development for teachers, it is hence crucial to also conduct research on different aspects of programming in the classroom. In this paper, we describe the process of developing a scientifically grounded instrument for assessing students' programming skills, as part of a longitudinal research project investigating how students in primary school learn programming. We also present the main findings related to the suitability of the instrument based on a pilot study conducted in spring 2019, collecting data from 310 students.

References

[1]
A. Balanskat and K. Engelhardt. Computing our future. computer programming and coding. priorities, school curricula and initiatives across Europe, 2015.
[2]
A. Bandura. Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1):1--26, 2002.
[3]
Barefoot CAS. The computational thinker: Concepts and approaches, 2016. http://barefootcas.org.uk. Accessed on March 7, 2019.
[4]
J. Bennedsen and M. E. Caspersen. An investigation of potential success factors for an introductory model-driven programming course. In Proceedings of the First International Workshop on Computing Education Research, ICER '05, page 155--163, New York, NY, USA, 2005. Association for Computing Machinery.
[5]
J. Boustedt, A. Eckerdal, R. McCartney, J. E. Moström, M. Ratcliffe, K. Sanders, and C. Zander. Threshold concepts in computer science: Do they exist and are they useful? In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education, SIGCSE '07, page 504--508, New York, NY, USA, 2007. Association for Computing Machinery.
[6]
K. Brennan and M. Resnick. New frameworks for studying and assessing the development of computational thinking. In Proc. of the Annual Meeting of the American Educational Research Assocication, 2012.
[7]
P. Bricheno and M. Thornton. Role model, hero or champion? children's views concerning role models. Educational Research, 49(4):383--396, 2007.
[8]
S. Cassidy and P. Eachus. Developing the computer self-efficacy (cse) scale: Investigating the relationship between cse, gender and experience with computers. Journal of Educational Computing Research, 26(2), 2002.
[9]
I. Cetin and M.Y. Ozden. Development of computer programming attitude scale for university students. Computer Applications in Engineering Education, 23(5):667--672, 2015.
[10]
C. Duncan and T. Bell. A pilot computer science and programming course for primary school students. In Proceedings of the Workshop in Primary and Secondary Computing Education, WiPSCE '15, page 39--48, New York, NY, USA, 2015. Association for Computing Machinery.
[11]
R. Eglash, J. E. Gilbert, and E. Foster. Toward culturally responsive computing education. Commun. ACM, 56(7):33--36, July 2013.
[12]
D. Fields, D. Lui, and Y. Kawai. Teaching computational thinking with electronic textiles: Modeling iterative practices and supporting personal projects in exploring computer science. Computational Thinking Education, pages 279--294, 2019.
[13]
S. Grover. Assessing algorithmic and computational thinking in k-12: Lessons from a middle school classroom. In P. Rich and C. Hodges, editors, Emerging Research, Practice, and Policy on Computational Thinking, pages 269--288. Springer, 2017.
[14]
S. Grover, S. Basu, and M. Bienkowski. Designing programming tasks for measuring computational thinking. In American Educational Research Association, 2017.
[15]
S. Grover and R. Pea. Computational thinking in k-12 a review of the state of the field. Educational Researcher, 42:38--43, 2013.
[16]
S. Grover and R. Pea. Designing a blended, middle school computer science course for deeper learning: A design-based research approach. In The International Conference of the Learning Sciences (ICLS)., 2016.
[17]
S. Grover, R. Pea, and S. Cooper. Factors influencing computer science learning in middle school. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education, SIGCSE '16, page 552--557, New York, NY, USA, 2016. Association for Computing Machinery.
[18]
S. Guggemos, S. Seufert, and M. Roman-Gonzalez. Measuring computational thinking - adapting a performance test and a self-assessment instrument for german-speaking countries. In 16th International Conference on Cognition and Exploratory Learning in Digital Age (CELDA'2019), 2019.
[19]
L. Hall, C. Hume, and S. Tazzyman. Five degrees of happiness: Effective smiley face likert scales for evaluating with children. In Proceedings of IDC 2016, pages 311--321, 2016.
[20]
F. Heintz and L. Mannila. Computational thinking for all: An experience report on scaling up teaching computational thinking to all students in a major city in sweden. ACM Inroads, 9(2):65--71, Apr. 2018.
[21]
F. Heintz, L. Mannila, and T. Färnqvist. A review of models for introducing computational thinking, computer science and computing in k-12 education. In 2016 IEEE Frontiers in Education Conference (FIE), pages 1--9, 2016.
[22]
T. Irish and N.-H. Kang. Connecting classroom science with everyday life: Teachers' attempts and students' insights. International Journal of Science and Mathematics Education, 16:1227--1245, 2018.
[23]
P. Kampylis and Y. P. (editors). Developing computational thinking in compulsory education - implications for policy and practice, 2016. JRC Science for Policy Report.
[24]
E. Kang, C. Donovan, and M. McCarthy. Exploring elementary teachers' pedagogical content knowledge and confidence in implementing the ngss science and engineering practices. Journal of Science Teacher Education, 29(1):9--29, 2018.
[25]
L. Leifheit, K. Tsarava, K. Moeller, K. Ostermann, J. Golle, U. Trautwein, and M. Ninaus. Development of a questionnaire on self-concept, motivational beliefs, and attitude towards programming. In Proceedings of the 14th Workshop in Primary and Secondary Computing Education, WiPSCE'19, New York, NY, USA, 2019. Association for Computing Machinery.
[26]
L. Mannila, L. Åke Nordén, and A. Pears. Digital competence, teacher self-efficacy and training needs. In Proceedings of ICER 2018, 2018.
[27]
Y. Maruyama. Investigation into parents' concerns about the introduction of programming education into japanese primary school. Procedia Computer Sciencen, 126:1037--1045, 2018.
[28]
M. McCracken, V. Almstrum, D. Diaz, M. Guzdial, D. Hagan, Y. B.-D. Kolikant, C. Laxer, L. Thomas, I. Utting, and T. Wilusz. A multi-national, multi-institutional study of assessment of programming skills of first-year cs students. In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education, ITiCSE-WGR '01, page 125--180, New York, NY, USA, 2001. Association for Computing Machinery.
[29]
J. Moreno-León, G. Robles, and M. Román-González. Dr. scratch: Automatic analysis of scratch projects to assess and foster computational thinking. In RED-Revista de Educación a Distancia, 2015.
[30]
A. Palmer. Hur blir man matematisk? :att skapa nya relationer till matematik och genus i arbetet med yngre barn. Stockholm: Liber, 2011.
[31]
M. Parker, M. Guzdial, and S. Engleman. Replication, validation, and use of a language independent cs1 knowledge assessment. In ICER 2016, 2016.
[32]
A. Pears, S. Seidman, L. Malmi, L. Mannila, E. Adams, J. Bennedsen, M. Devlin, and J. Paterson. A survey of literature on the teaching of introductory programming. SIGCSE Bull., 39(4):204--223, Dec. 2007.
[33]
PISA. Assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving. pisa 2015 collaborative problem-solving framework.
[34]
V. Ramalingam and S. Wiedenbeck. Development and validation of scores on a computer programming self-efficacy scale and group analyses of novice programmer self-efficacy. Journal of Educational Computing Research, 19(4):367--381, 1998.
[35]
M. Roman-Gonzalez. Computational thinking test: Design guidelines and content validation. In Proceedings of EDULEARN15 conference, pages 2436--2444, 2015.
[36]
Scratch team. Computational thinking with scratch - how do i assess the development of ct.
[37]
Simon, S. Fincher, A. Robins, B. Baker, I. Box, Q. Cutts, M. de Raadt, P. Haden, J. Hamer, M. Hamilton, R. Lister, M. Petre, K. Sutton, D. Tolhurst, and J. Tutty. Predictors of success in a first programming course. In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52, ACE '06, page 189--196, AUS, 2006. Australian Computer Society, Inc.
[38]
Skolverket. Läroplan för grundskolan, förskoleklassen och fritidshemmet: reviderad 2019, 2018.
[39]
X. Tang, Y. Yin, Q. Lin, R. Hadad, and X. Zhai. Assessing computational thinking: A systematic review of empirical studies. Computers and Education, 148, 2020.
[40]
A. E. Tew and M. Guzdial. Developing a validated assessment of fundamental cs1 concepts. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education, SIGCSE '10, page 97--101, New York, NY, USA, 2010. Association for Computing Machinery.
[41]
L. Vygotskij. Mind in society: the development of higher psychological processes. Cambridge, Mass.: Harvard U.P., 1978.
[42]
D. Weintrop, E. Beheshti, M. Horn, K. Orton, L. Trouille, K. Jona, and U. Wilensky. Interactive assessment tools for computational thinking in high school stem classrooms. In D. Reidsma, I. Choi, and R. Bargar, editors, Intelligent Technologies for Interactive Entertainment. INTETAIN 2014. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, volume 136, 2014.
[43]
D. Weintrop and U. Wilensky. Comparing block-based and text-based programming in high school computer science classrooms. ACM Trans. Comput. Educ., 18(1), Oct. 2017.
[44]
L. Werner, J. Denner, S. Campe, and D. C. Kawamoto. The fairy performance assessment: Measuring computational thinking in middle school. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education, SIGCSE '12, page 215--220, New York, NY, USA, 2012. Association for Computing Machinery.
[45]
S. Wiedenbeck. Factors affecting the success of non-majors in learning to program. In Proceedings of the First International Workshop on Computing Education Research, ICER '05, page 13--24, New York, NY, USA, 2005. Association for Computing Machinery.
[46]
C. Wilcox and A. Lionelle. Quantifying the benefits of prior programming experience in an introductory computer science course. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education, SIGCSE '18, page 80--85, New York, NY, USA, 2018. Association for Computing Machinery.
[47]
J. Wing. Computational thinking. Comm. of the ACM, 49(3):33--35, 2006.
[48]
M. Zapata, M. Estefania, and M. Roman-Gonzalez. Computational thinking test for beginners: Design and content validation. In IEEE Global Engineering Education Conference - EDUCON 2020, 2020.
[49]
B. Zimmerman. Self-efficacy: An essential motive to learn. Contemporary Educational Psychology, 25(1):82--91, 2000.

Cited By

View all
  • (2023)Introducing Programming Concepts Through the Bebras Tasks in the Primary EducationTeaching Coding in K-12 Schools10.1007/978-3-031-21970-2_10(145-156)Online publication date: 28-Feb-2023
  • (2022)Finnish teachers’ and students’ programming motivation and their role in teaching and learning computational thinkingFrontiers in Education10.3389/feduc.2022.9487837Online publication date: 21-Nov-2022
  • (2022)Assessment of Code, Which Aspects Do Teachers Consider and How Are They Valued?ACM Transactions on Computing Education10.1145/351713322:4(1-27)Online publication date: 15-Sep-2022
  • Show More Cited By

Index Terms

  1. Programming in primary education: towards a research based assessment framework

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      WiPSCE '20: Proceedings of the 15th Workshop on Primary and Secondary Computing Education
      October 2020
      179 pages
      ISBN:9781450387590
      DOI:10.1145/3421590
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 28 October 2020

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. K-12 education
      2. assessment
      3. programming

      Qualifiers

      • Research-article

      Funding Sources

      • Stiftelsen Marcus och Amalia Wallenbergs Minnesfond

      Conference

      WiPSCE '20

      Acceptance Rates

      Overall Acceptance Rate 104 of 279 submissions, 37%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)76
      • Downloads (Last 6 weeks)14
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Introducing Programming Concepts Through the Bebras Tasks in the Primary EducationTeaching Coding in K-12 Schools10.1007/978-3-031-21970-2_10(145-156)Online publication date: 28-Feb-2023
      • (2022)Finnish teachers’ and students’ programming motivation and their role in teaching and learning computational thinkingFrontiers in Education10.3389/feduc.2022.9487837Online publication date: 21-Nov-2022
      • (2022)Assessment of Code, Which Aspects Do Teachers Consider and How Are They Valued?ACM Transactions on Computing Education10.1145/351713322:4(1-27)Online publication date: 15-Sep-2022
      • (2022)Application Research of Computer-Assisted Primary School Mathematics Teaching2022 12th International Conference on Information Technology in Medicine and Education (ITME)10.1109/ITME56794.2022.00044(167-170)Online publication date: Nov-2022
      • (2022)Identifying Programming Skills Impacted in Students with Cognitive Disabilities2022 IEEE Frontiers in Education Conference (FIE)10.1109/FIE56618.2022.9962703(1-8)Online publication date: 8-Oct-2022
      • (2022)Exploring Gender Differences in Primary School Programming2022 IEEE Frontiers in Education Conference (FIE)10.1109/FIE56618.2022.9962482(1-9)Online publication date: 8-Oct-2022
      • (2021)Uses, Revisions, and the Future of Validated Assessments in Computing Education: A Case Study of the FCS1 and SCS1Proceedings of the 17th ACM Conference on International Computing Education Research10.1145/3446871.3469744(60-68)Online publication date: 16-Aug-2021
      • (2021)Computerized adaptive assessment of understanding of programming concepts in primary school childrenComputer Science Education10.1080/08993408.2021.191446132:4(418-448)Online publication date: 16-Apr-2021

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media