skip to main content
10.1145/3636555.3636913acmotherconferencesArticle/Chapter ViewAbstractPublication PageslakConference Proceedingsconference-collections
research-article

Quantifying Collaborative Complex Problem Solving in Classrooms using Learning Analytics

Published:18 March 2024Publication History

ABSTRACT

Complex problem solving (CPS) is a critical skill with far-reaching implications for personal success and professional development. While CPS research has made extensive progress, additional investigation is needed to explore CPS processes beyond online contexts and performance outcomes. This study, conducted with Year 9 students aged between thirteen and fourteen, focuses on collaborative CPS. It utilises audio and video recordings to capture group communications during a CPS classroom activity. To analyse these interactions, we introduce a novel CPS framework as a dynamic, cognitive and social process involving interrelated main skills, sub-skills, and indicators. Through sequential pattern mining, we identify recurring subskill patterns that reflect CPS processes in an educational environment. Our research underscores the importance of employing diverse patterns before plan execution, particularly building shared knowledge, planning, and negotiation. We uncover patterns related to groups going off-task and highlight the significance of effective communication and maintaining focus for keeping groups on track. Furthermore, we indicate patterns following the detection of emergent issues, recognising the value of cultivating clarity and adaptability among team members. Our CPS framework, combined with our research results, offers practical implications for teaching, learning, and assessment approaches in educational, professional and industry sectors.

References

  1. Fiore, S. M., Graesser, A. and Greiff, S. Collaborative problem-solving education for the twenty-first-century workforce. Nature Human Behaviour, 2, 6 (2018/06/01 2018), 367-369.Google ScholarGoogle ScholarCross RefCross Ref
  2. Funke, J., Fischer, A. and Holt, D. Competencies for Complexity: Problem Solving in the Twenty-First Century. City, 2018.Google ScholarGoogle Scholar
  3. Dörner, D. and Funke, J. Complex Problem Solving: What It Is and What It Is Not. Frontiers in Psychology, 8, 1153 (2017-July-11 2017).Google ScholarGoogle ScholarCross RefCross Ref
  4. OECD. PISA 2012 Problem-solving framework. . 2013.Google ScholarGoogle Scholar
  5. Neubert, J. C., Mainert, J., Kretzschmar, A. and Greiff, S. The Assessment of 21st Century Skills in Industrial and Organizational Psychology: Complex and Collaborative Problem Solving. Industrial and Organizational Psychology, 8, 2 (2015), 238-268.Google ScholarGoogle Scholar
  6. OECD. PISA 2012 Results: Creative Problem Solving: Students’ Skills in Tackling Real-Life Problems (Volume V), PISA, OECD Publishing. 2014.Google ScholarGoogle Scholar
  7. Schoppek, W. and Fischer, A. Complex problem solving—single ability or complex phenomenon? Frontiers in Psychology, 6, 1669 (2015-November-05 2015).Google ScholarGoogle ScholarCross RefCross Ref
  8. Herborn, K., Stadler, M., Mustafić, M. and Greiff, S. The assessment of collaborative problem solving in PISA 2015: Can computer agents replace humans? Computers in Human Behavior, 104 (2020/03/01/ 2020), 105624.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Martin-Raugh, M. P., Kyllonen, P. C., Hao, J., Bacall, A., Becker, D., Kurzum, C., Yang, Z., Yan, F. and Barnwell, P. Negotiation as an interpersonal skill: Generalizability of negotiation outcomes and tactics across contexts at the individual and collective levels. Computers in Human Behavior, 104 (2020/03/01/ 2020), 105966.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Stoeffler, K., Rosen, Y., Bolsinova, M. and von Davier, A. Gamified Performance Assessment of Collaborative Problem Solving Skills. Computers in Human Behavior, 104 (05/01 2019).Google ScholarGoogle Scholar
  11. Song, Y., Cheng, B., Zhu, J. and Hu, X. Exploring the collective process of classroom dialogue using sequential pattern mining technique. International Journal of Educational Research, 115 (2022/01/01/ 2022), 102050.Google ScholarGoogle ScholarCross RefCross Ref
  12. Liu, T. and Israel, M. Uncovering students’ problem-solving processes in game-based learning environments. Computers & Education, 182 (2022/06/01/ 2022), 104462.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. He, Q., Borgonovi, F. and Paccagnella, M. Leveraging process data to assess adults’ problem-solving skills: Using sequence mining to identify behavioral patterns across digital tasks. Computers & Education, 166 (2021/06/01/ 2021), 104170.Google ScholarGoogle ScholarCross RefCross Ref
  14. Zhang, Y. and Paquette, L. Sequential Pattern Mining in Educational Data: The Application Context, Potential, Strengths, and Limitations. Springer Nature Singapore, City, 2023.Google ScholarGoogle Scholar
  15. Zheng, J., Xing, W. and Zhu, G. Examining sequential patterns of self- and socially shared regulation of STEM learning in a CSCL environment. Computers & Education, 136 (2019/07/01/ 2019), 34-48.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Graesser, A., Kuo, B.-C. and Liao, C.-H. Complex Problem Solving in Assessments of Collaborative Problem Solving. Journal of Intelligence, 5, 2 (2017), 10.Google ScholarGoogle Scholar
  17. Zhu, G., Xing, W. and Popov, V. Uncovering the sequential patterns in transformative and non-transformative discourse during collaborative inquiry learning. The Internet and Higher Education, 41 (2019/04/01/ 2019), 51-61.Google ScholarGoogle ScholarCross RefCross Ref
  18. OECD. PISA 2015 Collborative problem-solving framework. 2017.Google ScholarGoogle Scholar
  19. Hesse, F., Care, E., Buder, J., Sassenberg, K. and Griffin, P. A Framework for Teachable Collaborative Problem Solving Skills. Springer Netherlands, City, 2015.Google ScholarGoogle Scholar
  20. Damşa, C. I. The multi-layered nature of small-group learning: Productive interactions in object-oriented collaboration. International Journal of Computer-Supported Collaborative Learning, 9, 3 (2014/09/01 2014), 247-281.Google ScholarGoogle ScholarCross RefCross Ref
  21. Sun, C., Shute, V. J., Stewart, A., Yonehiro, J., Duran, N. and D'Mello, S. Towards a generalized competency model of collaborative problem solving. Computers & Education, 143 (2020/01/01/ 2020), 103672.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Hesse, F., Care, E., Buder, J., Sassenberg, K. and Griffi, P. Chapter 2 A Framework for Teachable Collaborative Problem Solving Skills. City, 2017.Google ScholarGoogle Scholar
  23. Sun, C., Shute, V., Stewart, A., Beck-White, Q., Reinhardt, C., Zhou, G., Duran, N. and D'Mello, S. The relationship between collaborative problem solving behaviors and solution outcomes in a game-based learning environment. Computers in Human Behavior, 128 (11/01 2021), 107120.Google ScholarGoogle Scholar
  24. Fischer, A., Greiff, S. and Funke, J. The history of complex problem solving. City, 2017.Google ScholarGoogle Scholar
  25. Brennan, K. and Resnick, M. New frameworks for studying and assessing the development of computational thinking. City, 2012.Google ScholarGoogle Scholar
  26. Shute, V. J., Sun, C. and Asbell-Clarke, J. Demystifying computational thinking. Educational Research Review, 22 (2017/11/01/ 2017), 142-158.Google ScholarGoogle ScholarCross RefCross Ref
  27. Funke, J. Complex problem solving. Encyclopedia of the Sciences of Learning (682-685). Heidelberg: Springer (2012).Google ScholarGoogle Scholar
  28. Stoeffler, K., Rosen, Y., Bolsinova, M. and von Davier, A. A. Gamified performance assessment of collaborative problem solving skills. Computers in Human Behavior, 104 (2020/03/01/ 2020), 106036.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Bazerman, C., & Prior, P. (Eds.) What Writing Does and How It Does It: An Introduction to Analyzing Texts and Textual Practices (1st ed.). Routledge. , 2003.Google ScholarGoogle ScholarCross RefCross Ref
  30. Joksimović, S., Jovanović, J., Kovanović, V., Gašević, D., Milikić, N., Zouaq, A. and Staalduinen, J. P. v. Comprehensive Analysis of Discussion Forum Participation: From Speech Acts to Discussion Dynamics and Course Outcomes. IEEE Transactions on Learning Technologies, 13, 1 (2020), 38-51.Google ScholarGoogle ScholarCross RefCross Ref
  31. Strijbos, J.-W., Martens, R. L., Prins, F. J. and Jochems, W. M. G. Content analysis: What are they talking about? Computers & Education, 46, 1 (2006/01/01/ 2006), 29-48.Google ScholarGoogle ScholarCross RefCross Ref
  32. Agrawal, R. and Srikant, R. Mining Sequential Patterns. In Proceedings of the Proceedings of the Eleventh International Conference on Data Engineering (1995). IEEE Computer Society, [insert City of Publication],[insert 1995 of Publication].Google ScholarGoogle ScholarCross RefCross Ref
  33. Jian, P., Jiawei, H., Mortazavi-Asl, B., Jianyong, W., Pinto, H., Qiming, C., Dayal, U. and Mei-Chun, H. Mining sequential patterns by pattern-growth: the PrefixSpan approach. IEEE Transactions on Knowledge and Data Engineering, 16, 11 (2004), 1424-1440.Google ScholarGoogle Scholar
  34. Fournier Viger, P., Lin, C.-W., Rage, U., Koh, Y. S. and Thomas, R. A Survey of Sequential Pattern Mining. Data Science and Pattern Recognition, 1 (02/01 2017), 54-77.Google ScholarGoogle Scholar

Index Terms

  1. Quantifying Collaborative Complex Problem Solving in Classrooms using Learning Analytics
            Index terms have been assigned to the content through auto-classification.

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in
            • Published in

              cover image ACM Other conferences
              LAK '24: Proceedings of the 14th Learning Analytics and Knowledge Conference
              March 2024
              962 pages
              ISBN:9798400716188
              DOI:10.1145/3636555

              Copyright © 2024 ACM

              Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

              Publisher

              Association for Computing Machinery

              New York, NY, United States

              Publication History

              • Published: 18 March 2024

              Permissions

              Request permissions about this article.

              Request Permissions

              Check for updates

              Qualifiers

              • research-article
              • Research
              • Refereed limited

              Acceptance Rates

              Overall Acceptance Rate236of782submissions,30%
            • Article Metrics

              • Downloads (Last 12 months)71
              • Downloads (Last 6 weeks)71

              Other Metrics

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader

            HTML Format

            View this article in HTML Format .

            View HTML Format