skip to main content
research-article

The Landscape of Computational Thinking Problems for Practice and Assessment

Published: 14 March 2023 Publication History

Abstract

To provide practice and assessment of computational thinking, we need specific problems students can solve. There are many such problems, but they are hard to find. Learning environments and assessments often use only specific types of problems and thus do not cover computational thinking in its whole scope. We provide an extensive catalog of well-structured computational thinking problem sets together with a systematic encoding of their features. Based on this encoding, we propose a four-level taxonomy that provides an organization of a wide variety of problems. The catalog, taxonomy, and problem features are useful for content authors, designers of learning environments, and researchers studying computational thinking.

References

[1]
Charoula Angeli, Joke Voogt, Andrew Fluck, Mary Webb, Margaret Cox, Joyce Malyn-Smith, and Jason Zagami. 2016. A K-6 computational thinking curriculum framework: Implications for teacher knowledge. J. Educ. Technol. Soc. 19, 3 (2016), 47–57.
[2]
Tiffany Barnes. 2005. The q-matrix method: Mining student response data for knowledge. In American Association for Artificial Intelligence Educational Data Mining Workshop. AAAI Press, Washington, DC, 1–8.
[3]
Valerie Barr and Chris Stephenson. 2011. Bringing computational thinking to K-12: what is involved and what is the role of the computer science education community? ACM Inroads 2, 1 (2011), 48–54.
[4]
David Bau, Jeff Gray, Caitlin Kelleher, Josh Sheldon, and Franklyn Turbak. 2017. Learnable programming: Blocks and beyond. Commun. ACM 60, 6 (2017), 72–80.
[5]
Petr Boroš, Juraj Nižnan, Radek Pelánek, and Jiří Řihák. 2013. Automatic detection of concepts from problem solving times. In International Conference on Artificial Intelligence in Education. Springer, 595–598.
[6]
Acey Boyce and Tiffany Barnes. 2010. BeadLoom game: Using game elements to increase motivation and learning. In Proceedings of the 5th International Conference on the Foundations of Digital Games. 25–31.
[7]
Karen Brennan and Mitchel Resnick. 2012. New frameworks for studying and assessing the development of computational thinking. In Proceedings of the Annual Meeting of the American Educational Research Association, Vol. 1. 25.
[8]
Zack Butler, Ivona Bezáková, and Kimberly Fluet. 2017. Pencil puzzles for introductory computer science: An experience-and gender-neutral context. In Proceedings of the ACM SIGCSE Technical Symposium on Computer Science Education. 93–98.
[9]
Ünal Çakıroğlu, İsak Çevik, Engin Köşeli, and Merve Aydın. 2021. Understanding students’ abstractions in block-based programming environments: A performance based evaluation. Think. Skills Creativ. 41 (2021), 100888.
[10]
Patricia A. Carpenter, Marcel A. Just, and Peter Shell. 1990. What one intelligence test measures: A theoretical account of the processing in the raven progressive matrices test. Psychol. Rev. 97, 3 (1990), 404.
[11]
John Crabtree and Xihui Zhang. 2015. Recognizing and managing complexity: Teaching advanced programming concepts and techniques using the zebra puzzle. J. Inf. Technol. Educ.: Innov. Pract. 14 (2015), 171–189.
[12]
Valentina Dagienė and Gerald Futschek. 2008. Bebras international contest on informatics and computer literacy: Criteria for good tasks. In International Conference on Informatics in Secondary Schools-evolution and Perspectives. Springer, 19–30.
[13]
Valentina Dagienė and Sue Sentance. 2016. It’s computational thinking! bebras tasks in the curriculum. In International Conference on Informatics in Schools: Situation, Evolution, and Perspectives. Springer, 28–39.
[14]
Gabriella Daroczy, Magdalena Wolska, Walt Detmar Meurers, and Hans-Christoph Nuerk. 2015. Word problems: A review of linguistic and numerical factors contributing to their difficulty. Front. Psychol. 6 (2015), 348.
[15]
Javier del Olmo-Muñoz, Ramón Cózar-Gutiérrez, and José Antonio González-Calero. 2020. Computational thinking through unplugged activities in early years of primary education. Comput. Educ. 150 (2020), 103832.
[16]
Peter J. Denning and Matti Tedre. 2019. Computational Thinking. MIT Press.
[17]
Ruwan Devasurendra. 2020. Bebras Australia Computational Thinking Challenge.
[18]
Thomas M. Fiore, Alexander Lang, and Antonella Perucca. 2018. Tactile tools for teaching: Implementing knuth’s algorithm for mastering mastermind. Coll. Math. J. 49, 4 (2018), 278–286.
[19]
Lindsey Ann Gouws, Karen Bradshaw, and Peter Wentworth. 2013. Computational thinking in educational activities: An evaluation of the educational game light-bot. In Proceedings of the 18th ACM Conference on Innovation and Technology in Computer Science Education. 10–15.
[20]
Shuchi Grover and Roy Pea. 2018. Computational thinking: A competency whose time has come. In Computer Science Education: Perspectives on Teaching and Learning in School, Vol. 19.
[21]
Mariluz Guenaga, Andoni Eguíluz, Pablo Garaizar, and Juanjo Gibaja. 2021. How do students develop computational thinking? Assessing early programmers in a maze-based online game. Comput. Sci. Educ. 31, 2 (2021), 259–289.
[22]
Orit Hazzan, Tami Lapidot, and Noa Ragonis. 2014. Guide to Teaching Computer Science. Springer.
[23]
Sally A. M. Hogenboom, Felienne F. J. Hermans, and Han L. J. Van der Maas. 2021. Computerized adaptive assessment of understanding of programming concepts in primary school children. Comput. Sci. Educ. (2021), 1–30.
[24]
Juraj Hromkovič, Tobias Kohn, Dennis Komm, and Giovanni Serafini. 2016. Combining the power of python with the simplicity of logo for a sustainable computer science education. In International Conference on Informatics in Schools: Situation, Evolution, and Perspectives. Springer, 155–166.
[25]
Ting-Chia Hsu, Shao-Chen Chang, and Yu-Ting Hung. 2018. How to learn and how to teach computational thinking: Suggestions based on a review of the literature. Comput. Educ. 126 (2018), 296–310.
[26]
Lianne V. Hufkens and Cameron Browne. 2019. A functional taxonomy of logic puzzles. In Proceedings of the IEEE Conference on Games (CoG’19). IEEE, 1–4.
[27]
Nicholas Ichien, Hongjing Lu, and Keith J. Holyoak. 2020. Verbal analogy problem sets: An inventory of testing materials. Behav. Res. Methods 52, 5 (2020), 1803–1816.
[28]
Petri Ihantola, Tuukka Ahoniemi, Ville Karavirta, and Otto Seppälä. 2010. Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research. 86–93.
[29]
Petr Jarušek and Radek Pelánek. 2011. What determines difficulty of transport puzzles? In Proceedings of the 24th International FLAIRS Conference.
[30]
Filiz Kalelioglu, Yasemin Gülbahar, and Volkan Kukul. 2016. A framework for computational thinking based on a systematic research review. Balt. J. Mod. Comput. 4, 3 (2016), 583.
[31]
K. Schulz, S. Hobson, and J. Zagami. 2016. Bebras Australia computational thinking challenge - tasks and solution 2016. Australia: Digital Careers, Brisbane. https://www.csiro.au/-/media/Digital-Careers/Files/Bebras-Files/2016-Bebras-Solution-Guide-AU.pdf.
[32]
Caitlin Kelleher and Randy Pausch. 2005. Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers. ACM Comput. Surv. 37, 2 (2005), 83–137.
[33]
Yuhan Lin and David Weintrop. 2021. The landscape of Block-based programming: Characteristics of block-based environments and how they support the transition to text-based programming. J. Comput. Lang. 67 (2021), 101075.
[34]
Abraham S. Luchins. 1942. Mechanization in problem solving: The effect of einstellung. Psychol. Monogr. 54, 6 (1942), i.
[35]
Andrew Luxton-Reilly, Ibrahim Albluwi, Brett A. Becker, Michail Giannakos, Amruth N. Kumar, Linda Ott, James Paterson, Michael James Scott, Judy Sheard, and Claudia Szabo. 2018. Introductory programming: A systematic literature review. In Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education. 55–106.
[36]
Nicholas Lytle, Yihuan Dong, Veronica Cateté, Alex Milliken, Amy Isvik, and Tiffany Barnes. 2019. Position: Scaffolded coding activities afforded by block-based environments. In Proceedings of the IEEE Blocks and Beyond Workshop (B&B’19). IEEE, 5–7.
[37]
Thomas W. Malone and Mark R. Lepper. 2021. Making learning fun: A taxonomy of intrinsic motivations for learning. In Aptitude, Learning, and Instruction. Routledge, 223–254.
[38]
Linda Mannila, Valentina Dagiene, Barbara Demo, Natasa Grgurina, Claudio Mirolo, Lennart Rolandsson, and Amber Settle. 2014. Computational thinking in K-9 education. In Proceedings of the Working Group Reports of the on Innovation & Technology in Computer Science Education Conference. 1–29.
[39]
Jesús Moreno-León, Gregorio Robles, and Marcos Román-González. 2015. Dr. scratch: Automatic analysis of scratch projects to assess and foster computational thinking. Rev. Educ. Dist. 46 (2015), 1–23.
[40]
Greg L. Nelson, Benjamin Xie, and Andrew J. Ko. 2017. Comprehension first: Evaluating a novel pedagogy and tutoring system for program tracing in CS1. In Proceedings of the ACM Conference on International Computing Education Research. 2–11.
[41]
Seymour A. Papert. 2020. Mindstorms: Children, Computers, and Powerful Ideas. Basic Books.
[42]
Dale Parsons and Patricia Haden. 2006. Parson’s programming puzzles: A fun and effective learning tool for first programming courses. In Proceedings of the 8th Australasian Conference on Computing Education, Volume 52. 157–163.
[43]
Richard Pattis, J. Roberts, and M. Stehlik. 1981. Karel the robot. In A Gentle Introduction to the Art of Programming.
[44]
Radek Pelánek. 2017. Bayesian knowledge tracing, logistic models, and beyond: An overview of learner modeling techniques. User Model. User-Adapt. Interact. 27, 3 (2017), 313–350.
[45]
Radek Pelánek. 2020. A classification framework for practice exercises in adaptive learning systems. IEEE Trans. Learn. Technol. 13, 4 (2020), 734–747.
[46]
Radek Pelánek. 2021. Adaptive, intelligent, and personalized: Navigating the terminological maze behind educational technology. Int. J. Artif. Intell. Educ. 32, 1 (2021), 151--173.
[47]
Radek Pelánek and Tomáš Effenberger. 2022. Design and analysis of microworlds and puzzles for block-based programming. Comput. Sci. Educ. 32, 1 (2022), 66--104.
[48]
Yizhou Qian and James Lehman. 2017. Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Trans. Comput. Educ. 18, 1 (2017), 1–24.
[49]
Lloyd P. Rieber. 1996. Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educ. Technol. Res. Dev. 44, 2 (1996), 43–58.
[50]
Marcos Román-González, Jesús Moreno-León, and Gregorio Robles. 2019. Combining assessment tools for a comprehensive evaluation of computational thinking interventions. In Computational Thinking Education. Springer, Singapore, 79–98.
[51]
Marcos Román-González, Juan-Carlos Pérez-González, and Carmen Jiménez-Fernández. 2017. Which cognitive abilities underlie computational thinking? Criterion validity of the computational thinking test. Comput. Hum. Behav. 72 (2017), 678–691.
[52]
Elizabeth Rowe, Ma Victoria Almeda, Jodi Asbell-Clarke, Richard Scruggs, Ryan Baker, Erin Bardar, and Santiago Gasca. 2021. Assessing implicit computational thinking in zoombinis puzzle gameplay. Comput. Hum. Behav. 120 (2021), 106707.
[53]
Linda Seiter and Brendan Foreman. 2013. Modeling the learning progressions of computational thinking of primary grade students. In Proceedings of the 9th Annual International ACM Conference on International Computing Education Research. 59–66.
[54]
Sue Sentance, Jane Waite, and Maria Kallia. 2019. Teaching computer programming with PRIMM: A sociocultural perspective. Comput. Sci. Educ. 29, 2-3 (2019), 136–176.
[55]
Valerie J. Shute, Chen Sun, and Jodi Asbell-Clarke. 2017. Demystifying computational thinking. Educ. Res. Rev. 22 (2017), 142–158.
[56]
Helmut Simonis. 2005. Sudoku as a constraint problem. In CP Workshop on Modeling and Reformulating Constraint Satisfaction Problems, Vol. 12. 13–27.
[57]
Raymond Smullyan. 1986. What Is the Name of This Book? Touchstone Books.
[58]
Suwasono Suwasono, Dwi Prihanto, Irawan Dwi Wahyono, and Andrew Nafalski. 2017. Virtual laboratory for line follower robot competition. Int. J. Electr. Comput. Eng. 7, 4 (2017), 2253.
[59]
Alaaeddin Swidan, Felienne Hermans, and Marileen Smit. 2018. Programming misconceptions for school students. In Proceedings of the ACM Conference on International Computing Education Research. 151–159.
[60]
Xiaodan Tang, Yue Yin, Qiao Lin, Roxana Hadad, and Xiaoming Zhai. 2020. Assessing computational thinking: A systematic review of empirical studies. Comput. Educ. 148 (2020), 103798.
[61]
Matti Tedre and Peter J. Denning. 2016. The long quest for computational thinking. In Proceedings of the 16th Koli Calling International Conference on Computing Education Research. 120–129.
[62]
Peter C. Wason. 1960. On the failure to eliminate hypotheses in a conceptual task. Quart. J. Exp. Psychol. 12, 3 (1960), 129–140.
[63]
David Weintrop, Elham Beheshti, Michael Horn, Kai Orton, Kemi Jona, Laura Trouille, and Uri Wilensky. 2016. Defining computational thinking for mathematics and science classrooms. J. Sci. Educ. Technol. 25, 1 (2016), 127–147.
[64]
Greg Wilson. 2019. Teaching Tech Together: How to Make Your Lessons Work and Build a Teaching Community around Them. CRC Press.
[65]
Jeannette M. Wing. 2006. Computational thinking. Commun. ACM 49, 3 (2006), 33–35.
[66]
Jessica Yauney, Scott R. Bartholomew, and Peter Rich. 2022. A systematic review of “Hour of Code” research. Comput. Sci. Educ. (2022), 1–33.
[67]
LeChen Zhang and Jalal Nouri. 2019. A systematic review of learning computational thinking through Scratch in K-9. Comput. Educ. 141 (2019), 103607.

Cited By

View all
  • (2024)Bilgisayımsal Düşünme Becerilerinin Oyun Programlama Aracılığıyla Geliştirilmesi: Ortaokul Öğrencileri için Bir ÇerçeveAhi Evran Üniversitesi Sosyal Bilimler Enstitüsü Dergisi10.31592/aeusbed.1444312Online publication date: 30-Jul-2024
  • (2024)Experimental Analysis of First-Grade Students' Block-Based Programming Problem Solving ProcessesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653586(143-149)Online publication date: 3-Jul-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computing Education
ACM Transactions on Computing Education  Volume 23, Issue 2
June 2023
364 pages
EISSN:1946-6226
DOI:10.1145/3587033
  • Editor:
  • Amy J. Ko
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 March 2023
Online AM: 22 December 2022
Accepted: 09 December 2022
Revised: 01 December 2022
Received: 28 January 2022
Published in TOCE Volume 23, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Skills
  2. well-structured problems
  3. taxonomy

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)229
  • Downloads (Last 6 weeks)16
Reflects downloads up to 18 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Bilgisayımsal Düşünme Becerilerinin Oyun Programlama Aracılığıyla Geliştirilmesi: Ortaokul Öğrencileri için Bir ÇerçeveAhi Evran Üniversitesi Sosyal Bilimler Enstitüsü Dergisi10.31592/aeusbed.1444312Online publication date: 30-Jul-2024
  • (2024)Experimental Analysis of First-Grade Students' Block-Based Programming Problem Solving ProcessesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653586(143-149)Online publication date: 3-Jul-2024

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media