Abstract
As computational thinking (CT) gains more attention in K-16 education, problem-solving has been more emphasized as a core competency that can be found across various domains. To develop an evaluation framework that reveals students’ problem-solving competency, this study examined solutions for the Bebras Computing Challenge which requires students to utilize problem-solving skills in a CT domain. A total of 246 solutions of three Bebras tasks were analyzed based on a qualitative content analysis method and four levels of solutions were identified. The solution levels revealed how students (1) failed to understand a problem (No solution), (2) solved the problem but failed to identify the pattern (Premature level), (3) identified principles embedded in the problem but failed to apply them to devise an automized solution (Intermediate level), and (4) identified principles and solved the problem by applying them (Advanced level). This study presented solution levels across Bebras tasks and discussed how task difficulty affected student solutions differently. Implications for teaching problem-solving skills were discussed.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
No data is publicly available.
References
Allsop, Y. (2019). Assessing computational thinking process using a multiple evaluation approach. International Journal of Child-Computer Interaction, 19, 30–55. https://doi.org/10.1016/j.ijcci.2018.10.004
Basu, S., Dukeman, A., Kinnebrew, J. S., Biswas, G., & Sengupta, P. (2014). Investigating student generated computational models of science. Paper presented at the International Society of the Learning Sciences.
Bhagat, K. K., & Spector, J. M. (2017). Formative assessment in complex problem-solving domains: The emerging role of assessment technologies. Journal of Educational Technology & Society, 20(4), 312–317. Retrieved April 22, 2021, http://www.jstor.org/stable/26229226
Bransford, J. D., & Stein, B. S. (1993). The IDEAL problem solver. (2nd ed.). Freeman.
Brush, T., Ottenbreit-Leftwich, A., Kwon, K., & Karlin, M. (2020). Implementing Socially Relevant Problem-Based Computer Science Curriculum at the Elementary Level: Students’ Computer Science Knowledge and Teachers’ Implementation Needs. Journal of Computers in Mathematics and Science Teaching, 39(2), 109–123. Retrieved April 22, 2021, from https://www.learntechlib.org/primary/p/210969/
Chi, M. T. H. (2006). Laboratory methods for assessing experts’ and novices’ knowledge. In K. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance. (pp. 167–184). Cambridge University Press.
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152. https://doi.org/10.1207/s15516709cog0502_2
Chiazzese, G., Arrigo, M., Chifari, A., Lonati, V., & Tosto, C. (2019). Educational robotics in primary school: measuring the development of computational thinking skills with the Bebras tasks. Informatics, 6(4), 43. Retrieved from https://doi.org/10.3390/informatics6040043.
Dagienė, V., & Futschek, G. (2008). Bebras international contest on informatics and computer literacy: Criteria for good tasks. In R. T. Mittermeir & M. M. Sysło (Eds.), International conference on informatics in secondary schools-evolution and perspectives. (pp. 19–30). Springer.
Dagienė, V., & Stupuriene, G. (2016). Bebras–a sustainable community building model for the concept based learning of informatics and computational thinking. Informatics in education, 15(1), 25–44. https://doi.org/10.15388/infedu.2016.02
Dolgopolovas, V., Jevsikova, T., Savulionienė, L., Dagienė, V. (2015) On evaluation of computational thinking of software engineering novice students. Proceedings of the IFIP TC3 working conference. A new culture of learning: computing and next generations, 90–99.
Duncan, C., & Bell, T. (2015). A pilot computer science and programming course for primary school students. Paper presented at the Workshop in Primary and Secondary Computing Education.
Fawcett, L. M., & Garton, A. F. (2005). The effect of peer collaboration on children’s problem-solving ability. British Journal of Educational Psychology, 75(2), 157–169. https://doi.org/10.1348/000709904X23411
Fees, R. E., Rosa, J. A. d., Durkin, S. S., Murray, M. M., & Moran, A. L. (2018). Unplugged Cybersecurity: An approach for bringing computer science into the classroom. International Journal of Computer Science Education in Schools, 2(1). https://doi.org/10.21585/ijcses.v2i1.21
Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College composition and communication, 32(4), 365–387. https://doi.org/10.2307/356600
Frank, B., Simper, N., & Kaupp, J. (2018). Formative feedback and scaffolding for developing complex problem solving and modelling outcomes. European Journal of Engineering Education, 43(4), 552–568. https://doi.org/10.1080/03043797.2017.1299692
Gick, M. L. (1986). Problem-Solving Strategies. Educational Psychologist, 21(1–2), 99–120. https://doi.org/10.1080/00461520.1986.9653026
Greiff, S., Wüstenberg, S., Csapó, B., Demetriou, A., Hautamäki, J., Graesser, A. C., & Martin, R. (2014). Domain-general problem solving skills and education in the 21st century. Educational Research Review, 13, 74–83. https://doi.org/10.1016/j.edurev.2014.10.002
Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 28(1), 127–138. https://doi.org/10.1207/s15516709cog2801_7
Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277–1288. https://doi.org/10.1177/1049732305276687
Kalelioğlu, F., Gülbahar, Y., & Madran, O. (2015). A Snapshot of the First Implementation of Bebras International Informatics Contest in Turkey. Paper presented at the International Conference on Informatics in Schools: Situation, Evolution, and Perspectives, Cham.
Kapur, M. (2016). Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning. Educational Psychologist, 51(2), 289–299. https://doi.org/10.1080/00461520.2016.1155457
Kapur, M., & Bielaczyc, K. (2012). Designing for Productive Failure. Journal of the Learning Sciences, 21(1), 45–83. https://doi.org/10.1080/10508406.2011.591717
Kitchner, K. S. (1983). Cognition, Metacognition, and Epistemic Cognition. Human Development, 26(4), 222–232. https://doi.org/10.1159/000272885
Korkmaz, Ö., Çakir, R., & Özden, M. Y. (2017). A validity and reliability study of the computational thinking scales (CTS). Computers in Human Behavior, 72, 558–569. https://doi.org/10.1016/j.chb.2017.01.005
Krawec, J. L. (2014). Problem Representation and Mathematical Problem Solving of Students of Varying Math Ability. Journal of Learning Disabilities, 47(2), 103–115. https://doi.org/10.1177/0022219412436976
Lamagna, E. A. (2015). Algorithmic thinking unplugged. Journal of Computing Sciences in Colleges, 30(6), 45–52
Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Models of competence in solving physics problems. Cognitive Science, 4(4), 317–345. https://doi.org/10.1016/S0364-0213(80)80008-5
Polya, G. (1945). How to solve it. Princeton University Press.
Reilly, C. M., Kang, S. Y., Grotzer, T. A., Joyal, J. A., & Oriol, N. E. (2019). Pedagogical moves and student thinking in technology-mediated medical problem-based learning: Supporting novice-expert shift. British Journal of Educational Technology, 50(5), 2234–2250. https://doi.org/10.1111/bjet.12843
Renkl, A., & Atkinson, R. K. (2010). Learning from worked-out examples and problem solving. Cognitive load theory. (pp. 91–108). Cambridge University Press.
Repenning, A. (2017). Moving Beyond Syntax: Lessons from 20 Years of Blocks Programing in AgentSheets. Journal of Visual Languages and Sentient Systems, 3, 68–91. https://doi.org/10.18293/VLSS2017-010
Román-González, M., Pérez-González, J.-C., & Jiménez-Fernández, C. (2017). Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Human Behavior, 72, 678–691. https://doi.org/10.1016/j.chb.2016.08.047
Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3(4), 207–218. https://doi.org/10.1111/j.1467-9280.1992.tb00029.x
Schoenfeld, A. H. (1985). Mathematical problem-solving. Academic Press.
Thompson, V. A., Prowse Turner, J. A., & Pennycook, G. (2011). Intuition, reason, and metacognition. Cognitive Psychology, 63(3), 107–140. https://doi.org/10.1016/j.cogpsych.2011.06.001
Vygotsky, L. S. (1978). Mind in society. Harvard University Press.
Wang, Y., & Chiew, V. (2010). On the cognitive process of human problem solving. Cognitive Systems Research, 11(1), 81–92. https://doi.org/10.1016/j.cogsys.2008.08.003
Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. https://doi.org/10.1145/1118178.1118215
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
The study was approved by the University Institutional Review Board (IRB2018-1056) and did not involve any monetary compensation.
Conflict of interest
No potential competing interest was reported by the authors.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Bebras tasks
Appendix: Bebras tasks
1.1 Cipher wheel
A secret message was left on a beaver’s gravestone by using a cipher wheel and we want to find out what it means.
The wheel works such that only the inner wheel (with small letters) can be rotated. The outer wheel is for the actual message.
As you can see in the first image, when the key is 0 ‘A’ is encoded as ‘a.’
The second image shows that when the key is 17 (because the inner wheel has been rotated by 17 positions counter-clockwise) 'A' is encoded as 'r'.
With the key equal to 17, we can encode the message “WHO ARE YOU” as “nyf riv pfl” The message “mgvw ny twao” is received. We know that this was encrypted in a clever way: For the first letter the key was 1, for the second letter the key was 2, the key for the third letter was 3, etc. For instance, “sgg” with the same encryption method would be “RED”.
Question:
Decipher the encrypted message and choose the original message. (PLEASE attach a picture of problem-solving procedure/your work. It could be handwritten or digital).
-
a.
LOVE IS HERE
-
b.
LIFE IS GOOD
-
c.
LOVE IS MINE
-
d.
LESS IS MORE
1.2 Red Raider School
Red Raider School encourages its teachers to include games in their lessons.
One teacher invented the following game and he asks his students to play this game. The winner will leave school before dismissal.
Rules of the game:
The school has one hallway with four doors in a row. The students form a queue and take turns to walk down the hallway. When they get to an open door, they must close it and move to the next door. When they get to a closed door, they must open it, go into the classroom, leave the door open and wait there until the teacher dismisses them.
At the start of the game all the doors are closed. For example,
Start: | All doors are closed |
1st student: | The first is closed; open and enter |
2nd student: | Shut the first door, the second is closed, open and enter |
3rd student: | The first is closed, open and enter |
… |
If a student finds all the doors open, he or she will shut all of them, and leave school early!
Question:
If the students are numbered 1 to 20, which student gets to leave school first? (PLEASE attach a picture of problem-solving procedure/your work. It could be handwritten or digital) *HINT: Use a binary system.
-
a.
15th student
-
b.
16th student
-
c.
17th student
-
d.
18th student
1.3 Ballroom dance partners
Andy, Bert, Chris, David, and Eric are professional male ballroom dancers that take part in a TV show. Amy, Brenda, Carol, Dianna, and Emma are female participants that will learn to dance during this show. Each dancer will be assigned a single participant to teach.
Before the show, the producer organizes a party where everybody meets. After the party, the professionals and participants fill out a questionnaire:
-
each professional dancer ranks the participants in the order that he thinks they can be successful
-
each participant ranks the professionals in the order of how fast she can learn from him (1 = 1st choice, 2 = 2nd choice, etc.)
Here are the results of these choices:
Professional Dancers' Preferences | Participants' preferences | ||||||||||
Amy | Brenda | Carol | Dianna | Emma | Andy | Bert | Chris | David | Eric | ||
Andy | 2 | 1 | 4 | 5 | 3 | Amy | 5 | 2 | 3 | 1 | 4 |
Bert | 4 | 5 | 2 | 3 | 1 | Brenda | 1 | 3 | 5 | 2 | 4 |
Chris | 5 | 4 | 3 | 2 | 1 | Carol | 3 | 4 | 1 | 5 | 2 |
David | 1 | 3 | 2 | 5 | 4 | Dianna | 2 | 4 | 1 | 5 | 3 |
Eric | 3 | 5 | 1 | 2 | 4 | Emma | 5 | 2 | 3 | 4 | 1 |
The producers want to match the professionals with their ideal participants so that every participant is satisfied with his/her choice. You are asked to match the professionals with the contestants so that everyone has a perfect partner. You must also make sure that all unmatched pairs would still be happy with their partners.
Question:
When you have finished assigning partners, who is Eric's partner? (PLEASE attach a picture of problem-solving procedure/your work. It could be handwritten or digital).
-
a.
Amy
-
b.
Brenda
-
c.
Carol
-
d.
Dianna
Rights and permissions
About this article
Cite this article
Kwon, K., Cheon, J. & Moon, H. Levels of problem-solving competency identified through Bebras Computing Challenge. Educ Inf Technol 26, 5477–5498 (2021). https://doi.org/10.1007/s10639-021-10553-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-021-10553-9