Abstract
Computational thinking (CT) is an important 21st-century skill. This paper aims at more useful CT assessment. Available evaluation instruments are reviewed; two generally accepted CT evaluation tools are selected for a comprehensive CT assessment: the CTt, a performance test, and the CTS, a self-assessment instrument. The sample comprises 202 high school students from German-speaking Switzerland. Concerning the CTt, Rasch-scalability is demonstrated. Utilizing the approach of the PISA studies, proficiency levels are formed that comprise tasks with specific characteristics that students are systematically able to master. This could help teachers to offer individual support to their students. In terms of the CTS, the original version is refined using confirmatory factor and measurement-invariance analysis. A latent profile analysis yielded four profiles, two of which are of particular interest. One profile comprises students with, on the one hand, moderate to high creative thinking ability, cooperativity, and critical thinking skills and, on the other hand, low algorithmic thinking ability. The second remarkable profile consists of students with particularly low cooperativity. Based on these strength and weakness profiles, teachers could offer support tailored to student needs.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Readers who wish to examine the data may contact the corresponding author.
Notes
The German version of the CTt and CTS are available from the authors upon request.
The Rasch model predicts as the probability for a correct answer: exp(proficiency–difficulty) / (1 + exp(proficiency–difficulty)). For instance, a person with a proficiency of 1.5 Logit is expected to solve an item with a difficulty of 1.0 Logit with a probability of 62.2%.
Abbreviations
- aBIC:
-
Adjusted Bayesian information criterion
- AIC:
-
Akaike’s information criterion
- BIC:
-
Bayesian information criterion
- BLRT:
-
Bootstrap likelihood ratio test
- CAIC:
-
Consistent Akaike’s information criterion
- CFI:
-
Comparative fit index
- CI:
-
Confidence interval
- CT:
-
Computational thinking
- CTS:
-
Computational Thinking Scale
- CTt:
-
Computational Thinking Test
- DIF:
-
Differential item functioning
- EAP/PV:
-
Expected a posteriori / plausible values
- ICILS:
-
International Computer and Information Literacy Study
- LPA:
-
Latent profile analysis
- LRT:
-
Andersen’s likelihood ratio test
- MANOVA:
-
Multivariate analysis of variance
- MLR:
-
Maximum likelihood with robust standard errors
- PISA:
-
Programme for International Student Assessment
- R2 :
-
Coefficient of determination
- RMSEA:
-
Root mean square error of approximation
- RQ:
-
Research question
- SRMR:
-
Standardized root mean square residual
- TLI:
-
Tucker-Lewis Index
- WLE:
-
Weighted likelihood estimate
- WLSMV:
-
Diagonally weighted least squares
- wMNSQ:
-
Weighted mean square
- YB-χ2 :
-
Yuan-Bentler corrected χ2
- α:
-
Cronbach’s alpha
- ω:
-
Revell’s omega total
References
AERA, APA, & NCME. (2014). Standards for educational and psychological testing. American Educational Research Association. https://eric.ed.gov/?id=ED565876
Akogul, S., & Erisoglu, M. (2017). An approach for determining the number of clusters in a model-based cluster analysis. Entropy, 19(9), 452. https://doi.org/10.3390/e19090452
Andersen, E. B. (1973). A goodness of fit test for the Rasch model. Psychometrika, 38(1), 123–140. https://doi.org/10.1007/BF02291180
Barr, D., Harrison, J., & Conery, L. (2011). Computational thinking: A digital age skill for everyone. Learning & Leading with Technology, 38(6), 20–23. https://eric.ed.gov/?id=EJ918910
Brackmann, C. P., Román-González, M., Robles, G., Moreno-León, J., Casali, A., & Barone, D. (2017). Development of computational thinking skills through unplugged activities in primary school. In E. Barendsen (Ed.), Proceedings of the 12th Workshop on Primary and Secondary Computing Education (pp. 65–72). ACM. https://doi.org/10.1145/3137065.3137069
Bransford, J., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school (Expanded edition). National Academy Press. https://doi.org/10.17226/9853
Brennan, K., & Resnick, M. (2012). New frameworks for studying and assessing the development of computational thinking. American Educational Research Association Meeting, Vancouver, BC, Canada, 1–25. https://www.media.mit.edu/publications/new-frameworks-for-studying-and-assessing-the-development-of-computational-thinking/
Bühner, M. (2011). Einführung in die Test- und Fragebogenkonstruktion [Introduction to test and questionnaire construction] (3rd ed.). Pearson Studium. https://d-nb.info/1007197234
Buitrago Flórez, F., Casallas, R., Hernández, M., Reyes, A., Restrepo, S., & Danies, G. (2017). Changing a generation’s way of thinking: Teaching computational thinking through programming. Review of Educational Research, 87(4), 834–860. https://doi.org/10.3102/0034654317710096
Carretero, S., Vuorikari, R., & Punie, Y. (2017). DigComp 2.1: The Digital Competence Framework for Citizens with eight proficiency levels and examples of use. European Union (EU). https://doi.org/10.2760/38842
Chan, S.-W., Looi, C.-K., & Sumintono, B. (2020). Assessing computational thinking abilities among Singapore secondary students: A Rasch model measurement analysis. Journal of Computers in Education, 8, 213–236. https://doi.org/10.1007/s40692-020-00177-2
Chen, G., Shen, J., Barth-Cohen, L., Jiang, S., Huang, X., & Eltoukhy, M. (2017). Assessing elementary students’ computational thinking in everyday reasoning and robotics programming. Computers & Education, 109, 162–175. https://doi.org/10.1016/j.compedu.2017.03.001
Denner, J., Werner, L., Campe, S., & Ortiz, E. (2014). Pair programming: Under what conditions is it advantageous for middle school students? Journal of Research on Technology in Education, 46(3), 277–296. https://doi.org/10.1080/15391523.2014.888272
Doleck, T., Bazelais, P., Lemay, D. J., Saxena, A., & Basnet, R. B. (2017). Algorithmic thinking, cooperativity, creativity, critical thinking, and problem solving: Exploring the relationship between computational thinking skills and academic performance. Journal of Computers in Education, 4(4), 355–369. https://doi.org/10.1007/s40692-017-0090-9
Dorn, B., & Tew, A. E. (2015). Empirical validation and application of the computing attitudes survey. Computer Science Education, 25(1), 1–36. https://doi.org/10.1080/08993408.2015.1014142
Durak, H. Y., & Saritepeci, M. (2018). Analysis of the relation between computational thinking skills and various variables with the structural equation model. Computers & Education, 116, 191–202. https://doi.org/10.1016/j.compedu.2017.09.004
Embretson, S. E., & Daniel, R. C. (2008). Understanding and quantifying cognitive complexity level in mathematical problem solving items. Psychology Science Quarterly, 50(3), 328–344. https://www.semanticscholar.org/paper/Understanding-and-Quantifying-Cognitive-Complexity-Embretson-Daniel/29a75e2f74f0d5a870a846660c617a2273b73975
Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32(3), 179–186. https://doi.org/10.1080/00405849309543594
Ericson, B., & McKlin, T. (2012). Effective and sustainable computing summer camps. In L. Smith King, D. R. Musicant, T. Camp, & P. Tymann (Eds.), Proceedings of the 43rd ACM technical symposium on Computer Science Education - SIGCSE '12 (p. 289). ACM Press. https://doi.org/10.1145/2157136.2157223
Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37(6), 359–374. https://doi.org/10.1016/0001-6918(73)90003-6
Fishbein, M., & Ajzen, I. (2010). Predicting and changing behavior: The reasoned action approach. Taylor and Francis. https://psycnet.apa.org/record/2009-17267-000
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2019). Preparing for life in a digital world: IEA International Computer and Information Literacy Study 2018 international report. International Association for the Evaluation of Educational Achievement (IEA). https://www.iea.nl/publications/study-reports/preparing-life-digital-world
Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Duckworth, D. (2020). IEA International Computer and Information Literacy Study 2018: Technical Report. International Association for the Evaluation of Educational Achievement (IEA). https://www.iea.nl/publications/technical-reports/icils-2018-technical-report
Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., & Csapó, B. (2013). Complex problem solving in educational contexts—Something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105(2), 364–379. https://doi.org/10.1037/a0031856
Grover, S., & Pea, R. (2013). Computational thinking in K-12: A review of the state of the field. Educational Researcher, 42(1), 38–43. https://doi.org/10.3102/0013189X12463051
Grover, S., Pea, R., & Cooper, S. (2015). Designing for deeper learning in a blended computer science course for middle school students. Computer Science Education, 25(2), 199–237. https://doi.org/10.1080/08993408.2015.1033142
Grover, S., Pea, R., & Cooper, S. (2016). Factors influencing computer science learning in middle school. In C. Alphonce, J. Tims, M. Caspersen, & S. Edwards (Eds.), Proceedings of the 47th ACM Technical Symposium on Computing Science Education (pp. 552–557). ACM. https://doi.org/10.1145/2839509.2844564
Guggemos, J. (2021). On the predictors of computational thinking and its growth at the high-school level. Computers & Education. https://doi.org/10.1016/j.compedu.2020.104060
Guggemos, J., Seufert, S., & Román-González, M. (2019). Measuring computational thinking - Adapting a performance test and a self-assessment instrument for German-speaking countries. Proceedings of the 16th International Conference Cognition and Exploratory Learning in the Digital Age (CELDA) (pp. 183–191). https://doi.org/10.33965/celda2019_201911L023
Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2–24. https://doi.org/10.1108/EBR-11-2018-0203
Hallquist, M. N., & Wiley, J. F. (2018). MplusAutomation: An R package for facilitating large-scale latent variable analyses in Mplus. Structural Equation Modeling: A Multidisciplinary Journal, 25(4), 621–638. https://doi.org/10.1080/10705511.2017.1402334
Harteis, C., Goller, M., & Caruso, C. (2020). Conceptual change in the face of digitalization: Challenges for workplaces and workplace learning. Frontiers in Education. https://doi.org/10.3389/feduc.2020.00001
Hartig, J., & Frey, A. (2013). Sind modelle der item-response-theorie (IRT) das „Mittel der Wahl“ für die Modellierung von Kompetenzen? [Benefits and limitations of modeling competencies by means of Item Response Theory (IRT)]. Zeitschrift Für Erziehungswissenschaft, 16(S1), 47–51. https://doi.org/10.1007/s11618-013-0386-0
Hartig, J., Frey, A., Nold, G., & Klieme, E. (2012). An application of explanatory item response modeling for model-based proficiency scaling. Educational and Psychological Measurement, 72(4), 665–686. https://doi.org/10.1177/0013164411430707
Hofmans, J., Wille, B., & Schreurs, B. (2020). Person-centered methods in vocational research. Journal of Vocational Behavior, 118, 103398. https://doi.org/10.1016/j.jvb.2020.103398
Hooshyar, D., Malva, L., Yang, Y., Pedaste, M., Wang, M., & Lim, H. (2021). An adaptive educational computer game: Effects on students’ knowledge and learning attitude in computational thinking. Computers in Human Behavior, 114, 106575. https://doi.org/10.1016/j.chb.2020.106575
Hsu, T.-C., Chang, S.-C., & Hung, Y.-T. (2018). How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Computers & Education, 126, 296–310. https://doi.org/10.1016/j.compedu.2018.07.004
Ifenthaler, D., Hofhues, S., Egloffstein, M., & Helbig, C. (Eds.). (2021). Digital Transformation of Learning Organizations. Springer. https://doi.org/10.1007/978-3-030-55878-9
Ilic, U., Haseski, H. I., & Tugtekin, U. (2018). Publication trends over 10 years of computational thinking research. Contemporary Educational Technology., 9(2), 131–153. https://doi.org/10.30935/cet.414798
Israel, M., Pearson, J. N., Tapia, T., Wherfel, Q. M., & Reese, G. (2015). Supporting all learners in school-wide computational thinking: A cross-case qualitative analysis. Computers & Education, 82, 263–279. https://doi.org/10.1016/j.compedu.2014.11.022
Israel-Fishelson, R., & Hershkovitz, A. (2022). Studying interrelations of computational thinking and creativity: A scoping review (2011–2020). Computers & Education, 176, 104353. https://doi.org/10.1016/j.compedu.2021.104353
ISTE. (2015). Computational thinking: leadership toolkit. https://www.iste.org/computational-thinking
Jiang, B., Zhao, W., Gu, X., & Yin, C. (2021). Understanding the relationship between computational thinking and computational participation: A case study from Scratch online community. Educational Technology Research and Development, 69(5), 2399–2421. https://doi.org/10.1007/s11423-021-10021-8
Kirschner, P. A., & Stoyanov, S. (2020). Educating youth for nonexistent/not yet existing professions. Educational Policy, 34(3), 477–517. https://doi.org/10.1177/0895904818802086
Köhler, C., & Hartig, J. (2017). Practical significance of item misfit in educational assessments. Applied Psychological Measurement, 41(5), 388–400. https://doi.org/10.1177/0146621617692978
Kong, S. C., & Wang, Y. Q. (2021). Item response analysis of computational thinking practices: Test characteristics and students’ learning abilities in visual programming contexts. Computers in Human Behavior, 122, 106836. https://doi.org/10.1016/j.chb.2021.106836
Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005
Korkmaz, Ö., & Bai, X. (2019). Adapting computational thinking scale (CTS) for Chinese high school students and their thinking scale skills level. Participatory Educational Research, 6(1), 10–26. https://doi.org/10.17275/per.19.2.6.1
Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7
Kong, S. C. (2019). Components and methods of evaluating computational thinking for fostering creative problem-solvers in senior primary school education. In S. C. Kong & H. Abelson (Eds.), Computational thinking education (pp. 119–141). Springer. https://doi.org/10.1007/978-981-13-6528-7_8
Leys, C., Delacre, M., Mora, Y. L., Lakens, D., & Ley, C. (2019). How to classify, detect, and manage univariate and multivariate outliers, with emphasis on pre-registration. International Review of Social Psychology. https://doi.org/10.5334/irsp.289
Lohr, A., Stadler, M., Schultz-Pernice, F., Chernikova, O., Sailer, M., Fischer, F., & Sailer, M. (2021). On powerpointers, clickerers, and digital pros: Investigating the initiation of digital learning activities by teachers in higher education. Computers in Human Behavior, 119, 106715. https://doi.org/10.1016/j.chb.2021.106715
Lye, S. Y., & Koh, J. H. L. (2014). Review on teaching and learning of computational thinking through programming: What is next for K-12? Computers in Human Behavior, 41, 51–61. https://doi.org/10.1016/j.chb.2014.09.012
Mair, P., & Hatzinger, R. (2007) Extended Rasch modeling The eRm package for the application of IRT models in R. Journal of Statistical Software, 20(9), 1–20. https://doi.org/10.18637/jss.v020.i09
Maneesriwongul, W., & Dixon, J. K. (2004). Instrument translation process: A methods review. Journal of Advanced Nursing, 48(2), 175–186. https://doi.org/10.1111/j.1365-2648.2004.03185.x
McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144
Meyer, J. P., & Morin, A. J. (2016). A person-centered approach to commitment research: Theory, research, and methodology. Journal of Organizational Behavior, 37(4), 584–612. https://doi.org/10.1002/job.2085
Morin, A. J. S., & Marsh, H. W. (2015). Disentangling shape from level effects in person-centered analyses: An illustration based on university teachers’ multidimensional profiles of effectiveness. Structural Equation Modeling: A Multidisciplinary Journal, 22(1), 39–59. https://doi.org/10.1080/10705511.2014.919825
Mühling, A., Ruf, A., & Hubwieser, P. (2015). Design and first results of a psychometric test for measuring basic programming abilities. In A. Mühling, A. Ruf, & P. Hubwieser (Eds.), Design and First Results of a Psychometric Test for Measuring Basic Programming Abilities (pp. 2–10). ACM. https://doi.org/10.1145/2818314.2818320
OECD. (2015). PISA 2012 Technical Report. OECD Publishing. https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf
OECD. (2017). PISA 2015 Technical Report. OECD Publishing. http://www.oecd.org/pisa/sitedocument/PISA-2015-technical-report-final.pdf
Pellegrino, J. W., DiBello, L. V., & Goldman, S. R. (2016). A framework for conceptualizing and evaluating the validity of instructionally relevant assessments. Educational Psychologist, 51(1), 59–81. https://doi.org/10.1080/00461520.2016.1145550
Penfield, R. D., & Algina, J. (2006). A generalized DIF effect variance estimator for measuring unsigned differential test functioning in mixed format tests. Journal of Educational Measurement, 43(4), 295–312. https://doi.org/10.1111/j.1745-3984.2006.00018.x
Polat, E., Hopcan, S., Kucuk, S., & Sisman, B. (2021). A comprehensive assessment of secondary school students’ computational thinking skills. British Journal of Educational Technology, 52(5), 1965–1980. https://doi.org/10.1111/bjet.13092
Prenzel, M., Drechsel, B., & Kramer, K. (1998). Lernmotivation im kaufmännischen Unterricht: Die Sicht von Auszubildenden und Lehrkräften [Learning motivation in business education: the perspective of trainees and teachers]. Zeitschrift für Berufs- Und Wirtschaftspädagogik, special issue 14, 169–187
Relkin, E., de Ruiter, L. E., & Bers, M. U. (2021). Learning to code and the acquisition of computational thinking by young children. Computers & Education, 169, 104222. https://doi.org/10.1016/j.compedu.2021.104222
Repenning, A. (2006). Collaborative diffusion. Companion to the 21st ACM SIGPLAN Symposium on Object-Oriented Programming Systems, Languages, and Applications, 574–585. https://doi.org/10.1145/1176617.1176630
Repenning, A. (2017). Moving beyond syntax: Lessons from 20 years of blocks programing in AgentSheets. Journal of Visual Languages and Sentient Systems, 3(1), 68–91. https://doi.org/10.18293/VLSS2017-010
Repenning, A. (2018). Scale or fail: Moving beyond self-selected computer science education in Switzerland. Communications of the ACM, 61(5), 40–42. https://doi.org/10.1145/3199603
Repenning, A., Webb, D. C., Koh, K. H., Nickerson, H., Miller, S. B., Brand, C., Her Many Horses, I., Basawapatna, A., Gluck, F., Grover, R., Gutierrez, K., & Repenning, N. (2015). Scalable game design: A strategy to bring systemic computer science education to schools through game design and simulation creation. ACM Transactions on Computing Education, 15(2), 1–31. https://doi.org/10.1145/2700517
Resnick, M., & Rusk, N. (2020). Coding at a crossroads. Communications of the ACM, 63(11), 120–127. https://doi.org/10.1145/3375546
Richter, T., Naumann, J., & Horz, H. (2010). A revised version of the computer literacy inventory (INCOBI-R). Zeitschrift Für Pädagogische Psychologie, 24(1), 23–37. https://doi.org/10.1024/1010-0652/a000002
Robitzsch, A. (2020). Why ordinal variables can (almost) always be treated as continuous variables: Clarifying assumptions of robust continuous and ordinal factor analysis estimation methods. Frontiers in Education. https://doi.org/10.3389/feduc.2020.589965
Robitzsch, A., Kiefer, T., & Wu, M. (2020). Package 'TAM'. https://cran.r-project.org/web/packages/TAM/TAM.pdf
Román-González, M. (2015). Computational thinking test: Design guidlines and content validation. https://doi.org/10.13140/RG.2.1.4203.4329
Román-González, M., Pérez-González, J.-C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Computers in Human Behavior, 72, 678–691. https://doi.org/10.1016/j.chb.2016.08.047
Román-González, M., Moreno-León, J., & Robles, G. (2019). Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In S. C. Kong & H. Abelson (Eds.), Computational thinking education (pp. 79–98). Springer. https://doi.org/10.1007/978-981-13-6528-7_6
Rose, S. P., Habgood, M. J., & Jay, T. (2019). Using pirate plunder to develop children's abstraction skills in Scratch. In R. Mandryk, M. Hancock, M. Perry, & A. Cox (Eds.), Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–6). ACM Press. https://doi.org/10.1145/3290607.3312871
Rosenberg, J., Beymer, P., Anderson, D., van Lissa, C., & Schmidt, J. (2018). tidyLPA: An R package to easily carry out latent profile analysis (LPA) using open-source or commercial software. Journal of Open Source Software, 3(30), 978. https://doi.org/10.21105/joss.00978
Rosman, T., Mayer, A.-K., & Krampen, G. (2015). Combining self-assessments and achievement tests in information literacy assessment: Empirical results and recommendations for practice. Assessment & Evaluation in Higher Education, 40(5), 740–754. https://doi.org/10.1080/02602938.2014.950554
Rosseel, Y. (2012c). lavaan: An R package for structural equation modeling. Journal of Statistical Software., 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02
Salas, E., Reyes, D. L., & Woods, A. L. (2017). The assessment of team performance: observations and needs. In A. A. von Davier, M. Zhu, & P. C. Kyllonen (Eds.), Innovative assessment of collaboration (pp. 21–36). Springer. https://doi.org/10.1007/978-3-319-33261-1_2
Sarstedt, M., Becker, J.-M., Ringle, C. M., & Schwaiger, M. (2011). Uncovering and treating unobserved heterogeneity with FIMIX-PLS: Which model selection criterion provides an appropriate number of segments? Schmalenbach Business Review, 63(1), 34–62. https://doi.org/10.1007/BF03396886
Scherer, R., Howard, S. K., Tondeur, J., & Siddiq, F. (2021). Profiling teachers’ readiness for online teaching and learning in higher education: Who’s ready? Computers in Human Behavior, 118, 106675. https://doi.org/10.1016/j.chb.2020.106675
Scherer, R., Siddiq, F., & Sánchez Viveros, B. (2019). The cognitive benefits of learning computer programming: A meta-analysis of transfer effects. Journal of Educational Psychology, 111(5), 764–792. https://doi.org/10.1037/edu0000314
Scherer, R., Tondeur, J., & Siddiq, F. (2017). On the quest for validity: Testing the factor structure and measurement invariance of the technology-dimensions in the technological, pedagogical, and content knowledge (TPACK) model. Computers & Education, 112, 1–17. https://doi.org/10.1016/j.compedu.2017.04.012
Seiter, L., & Foreman, B. (2013). Modeling the learning progressions of computational thinking of primary grade students. In B. Simon, A. Clear, & Q. Cutts (Eds.), Proceedings of the ninth annual international ACM conference on International computing education research (pp. 59–66). ACM. https://doi.org/10.1145/2493394.2493403
Seufert, S., Guggemos, J., & Sailer, M. (2021). Technology-related knowledge, skills, and attitudes of pre- and in-service teachers: The current situation and emerging trends. Computers in Human Behavior, 115, 106552. https://doi.org/10.1016/j.chb.2020.106552
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017). Demystifying computational thinking. Educational Research Review, 22, 142–158. https://doi.org/10.1016/j.edurev.2017.09.003
Stadler, M., Herborn, K., Mustafić, M., & Greiff, S. (2020). The assessment of collaborative problem solving in PISA 2015: An investigation of the validity of the PISA 2015 CPS tasks. Computers & Education, 157, 103964. https://doi.org/10.1016/j.compedu.2020.103964
Tang, X., Yin, Y., Lin, Q., Hadad, R., & Zhai, X. (2020). Assessing computational thinking: A systematic review of empirical studies. Computers & Education, 148, 103798. https://doi.org/10.1016/j.compedu.2019.103798
Tondeur, J., Scherer, R., Baran, E., Siddiq, F., Valtonen, T., & Sointu, E. (2019). Teacher educators as gatekeepers: Preparing the next generation of teachers for technology integration in education. British Journal of Educational Technology, 50(3), 1189–1209. https://doi.org/10.1111/bjet.12748
Ulitzsch, E., von Davier, M., & Pohl, S. (2020). A hierarchical latent response model for inferences about examinee engagement in terms of guessing and item-level non-response. The British Journal of Mathematical and Statistical Psychology, 73(Suppl 1), 83–112. https://doi.org/10.1111/bmsp.12188
van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740
van Laar, E., van Deursen, A. J., van Dijk, J. A., & de Haan, J. (2017). The relation between 21st-century skills and digital skills: A systematic literature review. Computers in Human Behavior, 72, 577–588. https://doi.org/10.1016/j.chb.2017.03.010
Vanslambrouck, S., Zhu, C., Pynoo, B., Lombaerts, K., Tondeur, J., & Scherer, R. (2019). A latent profile analysis of adult students’ online self-regulation in blended learning environments. Computers in Human Behavior, 99, 126–136. https://doi.org/10.1016/j.chb.2019.05.021
Voogt, J., Fisser, P., Good, J., Mishra, P., & Yadav, A. (2015). Computational thinking in compulsory education: Towards an agenda for research and practice. Education and Information Technologies, 20(4), 715–728. https://doi.org/10.1007/s10639-015-9412-6
Werner, L., Denner, J., & Campe, S. (2012). The fairy performance assessment: measuring computational thinking in middle school. 43rd ACM Technical Symposium on Computer Science Education, 215–220. https://doi.org/10.1145/2157136.2157200
Weintrop, D., Wise Rutstein, D., Bienkowski, M., & McGee, S. (2021). Assessing computational thinking: An overview of the field. Computer Science Education, 31(2), 113–116. https://doi.org/10.1080/08993408.2021.1918380
Wilson, M. (2005). Constructing measures: An item response modeling approach. Psychology Press. https://psycnet.apa.org/record/2004-22396-000
Willse, J. T. (2011). Mixture rasch models with joint maximum likelihood estimation. Educational and Psychological Measurement, 71(1), 5–19. https://doi.org/10.1177/0013164410387335
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions Series a, Mathematical, Physical, and Engineering Sciences, 366(1881), 3717–3725. https://doi.org/10.1098/rsta.2008.0118
Yadav, A., Hong, H., & Stephenson, C. (2016). Computational thinking for all: Pedagogical approaches to embedding 21st century problem solving in K-12 classrooms. TechTrends, 60(6), 565–568. https://doi.org/10.1007/s11528-016-0087-7
Yadav, A., Mayfield, C., Zhou, N., Hambrusch, S., & Korb, J. T. (2014). Computational thinking in elementary and secondary teacher education. ACM Transactions on Computing Education, 14(1), 1–16. https://doi.org/10.1145/2576872
Zhao, W., & Shute, V. J. (2019). Can playing a video game foster computational thinking skills? Computers & Education, 141, 103633. https://doi.org/10.1016/j.compedu.2019.103633
Funding
The first author received a postdoctoral fellowship for carrying out this research from the Basic Research Fund at the University of St.Gallen (no. 1031542).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
No known conflicts of interest associated with this manuscript.
Ethical standards
The procedures performed in this study follow the standards of the Ethics Committee of the University of St.Gallen.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1: Excluded items due to psychometric validations
Appendix 2: Refined version of CTS
Creativity | cr_3 | “I believe that I can solve most of the problems I face if I have a sufficient amount of time and if I show effort.” |
cr_4 | “I believe that I can solve possible problems that may occur when I encounter a new situation.” | |
cr_5 | “I trust that I can apply a plan, at the same time as making it, in order to solve a problem.” | |
Algorithmic thinking | al_3 | “I think that I am better able to learn instructions with the help of mathematical symbols and concepts.” |
al_4 | “I can mathematically express the solutions for the problems I face in daily life.” | |
al_5 | “I can digitize a mathematical problem expressed verbally.” | |
Cooperativity | co_1 | “I like experiencing cooperative learning together in my group of friends.” |
co_2 | “In cooperative learning, I think that I attain/will attain more successful results because I am working in a group.” | |
co_3 | “I like solving problems related to a group project together with my friends in cooperative learning.” | |
Critical thinking | cr_1 | “I am willing to learn challenging things.” |
cr_2 | “I am proud of being able to think with great precision.” | |
cr_3 | “I make use of a systematic method while comparing the options at hand and while reaching a decision.” | |
Problem solving | pr_1 | “I have problems in demonstrating the solution to a problem in my mind.” (R) |
pr_2 | “I have difficulties regarding the issue of where and how I should use variables such as X and Y in the solution of a problem.” (R) | |
pr_4 | “I cannot apply the solutions I plan respectively and gradually.” (R) |
Appendix 3
Rights and permissions
About this article
Cite this article
Guggemos, J., Seufert, S. & Román-González, M. Computational Thinking Assessment – Towards More Vivid Interpretations. Tech Know Learn 28, 539–568 (2023). https://doi.org/10.1007/s10758-021-09587-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10758-021-09587-2