Skip to main content

Advertisement

SpringerLink
Log in

Using learning analytics to explore the multifaceted engagement in collaborative learning

  • Published:
Journal of Computing in Higher Education Aims and scope Submit manuscript

Abstract

Engagement is critical in learning, including computer-supported collaborative learning (CSCL). Previous studies have mainly measured engagement using students’ self-reports which usually do not capture the learning process or the interactions between group members. Therefore, researchers advocated developing new and innovative engagement measurements to address these issues through employing learning analytics and educational data mining (e.g., Azevedo in Educ Psychol 50(1):84–94, 2015; Henrie in Comput Educ 90:36–53, 2015). This study responded to this call by developing learning analytics to study the multifaceted aspects of engagement (i.e., group behavioral, social, cognitive, and metacognitive engagement) and its impact on collaborative learning. The results show that group behavioral engagement and group cognitive engagement have a significantly positive effect on group problem-solving performance; group social engagement has a significantly negative effect; the impact of group metacognitive engagement is not significant. Furthermore, group problem-solving performance has a significant positive effect on individual cognitive understanding, which partially mediates the impact of group behavioral engagement and fully mediates the impact of group social engagement on individual cognitive understanding. The findings have important implications for developing domain-specific learning analytics to measure students’ sub-constructs of engagement in CSCL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Akçapınar, G., Altun, A., & Aşkar, P. (2019). Using learning analytics to develop early-warning system for at-risk students. International Journal of Educational Technology in Higher Education, 16(1), 1–20.

    Article  Google Scholar 

  • Alwin, D. F., & Hauser, R. M. (1975). The decomposition of effects in path analysis. American Sociological Review. https://doi.org/10.2307/2094445

    Article  Google Scholar 

  • Arguedas, M., Daradoumis, T., & Xhafa, F. (2016). Analyzing how emotion awareness influences students’ motivation, engagement, self-regulation and learning outcome. Educational Technology & Society, 19(2), 87–103.

    Google Scholar 

  • Azevedo, R. (2015). Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist, 50(1), 84–94. https://doi.org/10.1080/00461520.2015.1004069

    Article  Google Scholar 

  • Baker, R., & Siemens, G. (2014). Educational data mining and learning analytics. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (2nd ed., pp. 253–274). Cambridge University Press.

    Chapter  Google Scholar 

  • Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173. https://doi.org/10.1037//0022-3514.51.6.1173

    Article  Google Scholar 

  • Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359. https://doi.org/10.1207/S15327809JLS1203_1

    Article  Google Scholar 

  • Chen, J., Wang, M., Kirschner, P. A., & Tsai, C.-C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584

    Article  Google Scholar 

  • Cocea, M., & Weibelzahl, S. (2011). Disengagement detection in online learning: Validation studies and perspectives. IEEE Transactions on Learning Technologies, 4, 114–124. https://doi.org/10.1109/TLT.2010.14

    Article  Google Scholar 

  • Csanadi, A., Eagan, B., Kollar, I., Shaffer, D. W., & Fischer, F. (2018). When coding-and-counting is not enough: Using epistemic network analysis (ENA) to analyze verbal data in CSCL research. International Journal of Computer-Supported Collaborative Learning, 13(4), 419–438. https://doi.org/10.1007/s11412-018-9292-z

    Article  Google Scholar 

  • Dalal, M. K., & Zaveri, M. A. (2011). Automatic text classification: A technical review. International Journal of Computer Applications, 28(2), 37–40. https://doi.org/10.5120/3358-4633

    Article  Google Scholar 

  • Dillenbourg, P. (1999). What do you mean by collaborative learning? In P. Dillenbourg (Ed.), Collaborative learning: Cognitive and computational approaches (pp. 1–19). Pergamon.

    Google Scholar 

  • Duit, R., & Treagust, D. (1998). Learning in science: From behaviorism towards social constructivism and beyond. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education. Dordrecht: Kluwer Academic Publishers.

    Google Scholar 

  • Engle, R. A., & Conant, F. R. (2002). Guiding principles for fostering productive disciplinary engagement: Explaining an emergent argument in a community of learners classroom. Cognition and Instruction, 20, 399–483. https://doi.org/10.1207/S1532690XCI2004_1

    Article  Google Scholar 

  • Erkens, G., Jaspers, J., Prangsma, M., & Kanselaar, G. (2005). Coordination processes in computer supported collaborative writing. Computers in Human Behavior, 21(3), 463–486. https://doi.org/10.1016/j.chb.2004.10.038

    Article  Google Scholar 

  • Fredricks, J., Blumenfeld, P., & Paris, P. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59–109. https://doi.org/10.3102/00346543074001059

    Article  Google Scholar 

  • Garson, G. D. (2013). Path analysis. Statistical Associates Publishing.

    Google Scholar 

  • Gijlers, H., & de Jong, T. (2009). Sharing and confronting propositions in collaborative learning. Cognition and Instruction, 27(3), 239–268. https://doi.org/10.1080/07370000903014352

    Article  Google Scholar 

  • Gobert, J. D., Baker, R. S., & Wixon, M. B. (2015). Operationalizing and detecting disengagement within online science microworlds. Educational Psychologist, 50(1), 43–57. https://doi.org/10.1080/00461520.2014.999919

    Article  Google Scholar 

  • Graesser, A. C., Fiore, S. M., Greiff, S., Andrews-Todd, J., Foltz, P. W., & Hesse, F. W. (2018). Advancing the science of collaborative problem solving. Psychological Science in the Public Interest, 19(2), 59–92. https://doi.org/10.1177/1529100618808244

    Article  Google Scholar 

  • Gresalfi, M., & Barab, S. (2010). Learning for a reason: Supporting forms of engagement by designing tasks and orchestrating environments. Theory into Practice, 50, 300–310. https://doi.org/10.1080/00405841.2011.607391

    Article  Google Scholar 

  • Gresalfi, M., Barab, S., Siyahhan, S., & Christensen, T. (2009). Virtual worlds, conceptual understanding, and me: Designing for consequential engagement. On the Horizon, 17, 21–34. https://doi.org/10.1108/10748120910936126

    Article  Google Scholar 

  • Griffin, P., McGaw, B., & Care, E. (2012). Assessment and teaching of 21st century skills. Melbourne, Australia: Springer. https://doi.org/10.1007/978-94-007-2324-5

  • Guertin, L. A., Bodek, M. J., Zappe, S. E., & Kim, H. (2007). Questioning the student use of and desire for lecture podcasts. MERLOT Journal of Online Learning and Teaching, 3(2), 133–141.

    Google Scholar 

  • Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In Proceedings of the First ACM Conference on Learning@ scale conference (pp. 41–50). ACM. https://doi.org/10.1145/2556325.2566239

  • Hasan, R., Palaniappan, S., Mahmood, S., Abbas, A., Sarker, K. U., & Sattar, M. U. (2020). Predicting student performance in higher educational institutions using video learning analytics and data mining techniques. Applied Sciences, 10(11), 3894.

    Article  Google Scholar 

  • Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-mediated learning: A review. Computers & Education, 90, 36–53. https://doi.org/10.1016/j.compedu.2015.09.005

    Article  Google Scholar 

  • Hmelo-Silver, C. E. (2003). Analyzing collaborative knowledge construction: Multiple methods for integrated understanding. Computers & Education, 41(4), 397–420. https://doi.org/10.1016/j.compedu.2003.07.001

    Article  Google Scholar 

  • Ifenthaler, D., & Widanapathirana, C. (2014). Development and validation of a learning analytics framework: Two case studies using support vector machines. Technology, Knowledge and Learning, 19(1–2), 221–240.

    Article  Google Scholar 

  • Jeong, H., Hmelo-Silver, C. E., Jo, K., & Shin, M. (2016). CSCL in STEM education: Preliminary findings from a meta-analysis. In 2016 49th Hawaii international conference on system sciences (HICSS) (pp. 11–20). IEEE. https://doi.org/10.1109/HICSS.2016.11

  • Jung, Y., & Lee, J. (2018). Learning engagement and persistence in massive open online courses (MOOCS). Computers & Education, 122, 9–22. https://doi.org/10.1016/j.compedu.2018.02.013

    Article  Google Scholar 

  • Kapur, M. (2008). Productive failure. Cognition and Instruction, 26(3), 379–424. https://doi.org/10.1080/07370000802212669

    Article  Google Scholar 

  • Kearsley, G., & Schneiderman, B. (1999). Engagement theory: A framework for technology-based learning and teaching. Educational Technology, 38(5), 20–23.

    Google Scholar 

  • Khalil, M., & Ebner, M. (2017). Clustering patterns of engagement in massive open online courses (MOOCs): The use of learning analytics to reveal student categories. Journal of Computing in Higher Education, 29(1), 114–132. https://doi.org/10.1007/s12528-016-9126-9

    Article  Google Scholar 

  • King, A. (1997). ASK to THINK-TELL WHY: A model of transactive peer tutoring for scaffolding higher level complex learning. Educational Psychologist, 32(4), 221–235. https://doi.org/10.1207/s15326985ep3204_3

    Article  Google Scholar 

  • King, A. (2002). Structuring peer interaction to promote high-level cognitive processing. Theory into Practice, 41(1), 33–39. https://doi.org/10.1207/s15430421tip4101_6

    Article  Google Scholar 

  • Kirschner, P. A., Beers, P. J., Boshuizen, H. P., & Gijselaers, W. H. (2008). Coercing shared knowledge in collaborative learning environments. Computers in Human Behavior, 24(2), 403–420. https://doi.org/10.1016/j.chb.2007.01.028

    Article  Google Scholar 

  • Kollar, I., Fischer, F., & Slotta, J. D. (2007). Internal and external scripts in computer-supported collaborative inquiry learning. Learning & Instruction, 17(6), 708–721. https://doi.org/10.1016/j.learninstruc.2007.09.021

    Article  Google Scholar 

  • Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., & Siemens, G. (2016). Towards automated content analysis of discussion transcripts: A cognitive presence case. In Proceedings of the sixth international conference on learning analytics & knowledge (pp. 15–24).

  • Kwon, K., Liu, Y. H., & Johnson, L. P. (2014). Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Computers & Education, 78, 185–200. https://doi.org/10.1016/j.compedu.2014.06.004

    Article  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. https://doi.org/10.2307/2529310

    Article  Google Scholar 

  • Linnenbrink-Garcia, L., Rogat, T. K., & Koskey, K. L. (2011). Affect and engagement during small group instruction. Contemporary Educational Psychology, 36(1), 13–24. https://doi.org/10.1016/j.cedpsych.2010.09.001

    Article  Google Scholar 

  • Long, P., & Siemen, G. (2011). Penetrating the fog: analytics in learning and education. EDUCAUSE Review, 46(5), 30–40. https://doi.org/10.17471/2499-4324/195

    Article  Google Scholar 

  • Lu, O. H., Huang, J. C., Huang, A. Y., & Yang, S. J. (2017). Applying learning analytics for improving students engagement and learning outcomes in an MOOCs enabled collaborative programming course. Interactive Learning Environments, 25(2), 220–234. https://doi.org/10.1080/10494820.2016.1278391

    Article  Google Scholar 

  • Monllaó Olivé, D., Huynh, D. Q., Reynolds, M., Dougiamas, M., & Wiese, D. (2020). A supervised learning framework: Using assessment to identify students at risk of dropping out of a MOOC. Journal of Computing in Higher Education, 32(1), 9–26. https://doi.org/10.1007/s12528-019-09230-1

    Article  Google Scholar 

  • Mullins, D., Rummel, N., & Spada, H. (2011). Are two heads always better than one? Differential effects of collaboration on students’ computer-supported learning in mathematics. International Journal of Computer-Supported Collaborative Learning, 6(3), 421–443. https://doi.org/10.1007/s11412-011-9122-z

    Article  Google Scholar 

  • Murugan, A., Nair, S. A. H., & Kumar, K. S. (2019). Detection of skin cancer using SVM, random forest and kNN classifiers. Journal of Medical Systems, 43(8), 1–9.

    Article  Google Scholar 

  • Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalized feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/bjet.12592

    Article  Google Scholar 

  • Pekrun, R., & Linnenbrink-Garcia, L. (2012). Academic emotions and student engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 259–282). Springer. https://doi.org/10.1007/978-1-4614-2018-7_12

    Chapter  Google Scholar 

  • Phan, T., McNeil, S. G., & Robin, B. R. (2016). Students’ patterns of engagement and course performance in a massive open online course. Computers & Education, 95, 36–44. https://doi.org/10.1016/j.compedu.2015.11.015

    Article  Google Scholar 

  • Pifarre, M., & Cobos, R. (2010). Promoting metacognitive skills through peer scaffolding in a CSCL environment. International Journal of Computer-Supported Collaborative Learning, 5(2), 237–253. https://doi.org/10.1007/s11412-010-9084-6

    Article  Google Scholar 

  • Ploetzner, R., Dillenbourg, P., Preier, M., & Traum, D. (1999). Learning by explaining to oneself and to others. Collaborative Learning: Cognitive and Computational Approaches, 1, 103–121.

    Google Scholar 

  • Popov, V., van Leeuwen, A., & Buis, S. C. A. (2017). Are you with me or not? Temporal synchronicity and transactivity during CSCL. Journal of Computer Assisted Learning, 33(5), 424–442. https://doi.org/10.1111/jcal.12185

    Article  Google Scholar 

  • Popov, V., Xing, W., Zhu, G., Horwitz, P., & McIntyre, C. (2018, June). The influence of students’ transformative and non-transformative contributions on their problem solving in collaborative inquiry learning. In The proceedings of the 13th international conference of the learning sciences (pp. 855–862). International Society of the Learning Sciences.

  • Richardson, J. C., Maeda, Y., Lv, J., & Caskurlu, S. (2017). Social presence in relation to students’ satisfaction and learning in the online environment: A meta-analysis. Computers in Human Behavior, 71, 402–417. https://doi.org/10.1016/j.chb.2017.02.001

    Article  Google Scholar 

  • Rizvi, S., Rienties, B., & Khoja, S. A. (2019). The role of demographics in online learning; A decision tree based approach. Computers & Education, 137, 32–47.

    Article  Google Scholar 

  • Rogat, T. K., & Adams-Wiggins, K. R. (2015). Interrelation between regulatory and socioemotional processes within collaborative groups characterized by facilitative and directive other-regulation. Computers in Human Behavior, 52, 589–600. https://doi.org/10.1016/j.chb.2015.01.026

    Article  Google Scholar 

  • Rogat, T. K., & Linnenbrink-Garcia, L. (2011). Socially shared regulation in collaborative groups: An analysis of the interplay between quality of social regulation and group processes. Cognition and Instruction, 29(4), 375–415. https://doi.org/10.1080/07370008.2011.607930

    Article  Google Scholar 

  • Shah, K., Patel, H., Sanghvi, D., & Shah, M. (2020). A comparative analysis of logistic regression, random forest and KNN models for the text classification. Augmented Human Research, 5(1), 1–16.

    Article  Google Scholar 

  • Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13. https://doi.org/10.1080/00461520.2014.1002924

    Article  Google Scholar 

  • Sinha, S., Rogat, T. K., Adams-Wiggins, K. R., & Hmelo-Silver, C. E. (2015). Collaborative group engagement in a computer-supported learning environment. International Journal of Computer-Supported Collaborative Learning, 10(3), 273–307. https://doi.org/10.1007/s11412-015-9218-y

    Article  Google Scholar 

  • Stahl, G., Koschmann, T., & Suthers, D. (2006). Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 409–426). Cambridge University Press.

    Google Scholar 

  • Stegmann, K., Kollar, I., Zottmann, J., Gijlers, H., De Jong, T., Dillenbourg, P., & Fischer, F. (2007). Towards the convergence of CSCL and learning: Scripting collaborative learning. In Proceedings of the 8th international conference on Computer-supported collaborative learning (pp. 831–832). International Society of the Learning Sciences.

  • Sun, J. C. Y., & Rueda, R. (2012). Situational interest, computer self-efficacy and self-regulation: Their impact on student engagement in distance education. British Journal of Educational Technology, 43(2), 191–204. https://doi.org/10.1111/j.1467-8535.2010.01157.x

    Article  Google Scholar 

  • Van den Bossche, P., Gijselaers, W. H., Segers, M., & Kirschner, P. A. (2006). Social and cognitive factors driving teamwork in collaborative learning environments team learning beliefs and behaviors. Small Group Research, 37, 490–521. https://doi.org/10.1177/1046496406292938

    Article  Google Scholar 

  • van Zyl, J. M., Neudecker, H., & Nel, D. G. (2000). On the distribution of the maximum likelihood estimator of Cronbach’s alpha. Psychometrika, 65(3), 271–280. https://doi.org/10.1007/BF02296146

    Article  Google Scholar 

  • Vogel, F., Wecker, C., Kollar, I., & Fischer, F. (2017). Socio-cognitive scaffolding with computer-supported collaboration scripts: A meta-analysis. Educational Psychology Review, 29(3), 477–511. https://doi.org/10.1007/s10648-016-9361-7

    Article  Google Scholar 

  • Vygotsky, L. S. (1978). Mind in society. Harvard University Press.

    Google Scholar 

  • Weinberger, A., Ertl, B., Fischer, F., & Mandl, H. (2005). Epistemic and social scripts in computer-supported collaborative learning. Instructional Science, 33(1), 1–30. https://doi.org/10.1007/s11251-004-2322-4

    Article  Google Scholar 

  • Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467. https://doi.org/10.1007/s11412-017-9267-5

    Article  Google Scholar 

  • Wiyono, S., Wibowo, D. S., Hidayatullah, M. F., & Dairoh, D. (2020). Comparative study of KNN, SVM and decision tree algorithm for student’s performance prediction. International Journal of Computing Science and Applied Mathematics, 6(2), 50–53.

    Article  Google Scholar 

  • Xing, W., Guo, R., Petakovic, E., & Goggins, S. (2015a). Participation-based student final performance prediction model through interpretable Genetic Programming: Integrating learning analytics, educational data mining and theory. Computers in Human Behavior, 47, 168–181. https://doi.org/10.1016/j.chb.2014.09.034

  • Xing, W., Popov, V., Zhu, G., Horwitz, P., & McIntyre, C. (2019). The effects of transformative and non-transformative discourse on individual performance in collaborative-inquiry learning. Computers in Human Behavior, 98, 267–276.

  • Xing, W., Wadholm, R., Petakovic, E., & Goggins, S. (2015b). Group learning assessment: Developing a theory-informed analytics. Journal of Educational Technology & Society, 18(2), 110–128. https://www.jstor.org/stable/jeductechsoci.18.2.110

  • Ye, D., & Pennisi, S. (2022). Using trace data to enhance students self-regulation: A learning analytics perspective. The Internet and Higher Education, 54, 100855. https://doi.org/10.1016/j.iheduc.2022.100855

    Article  Google Scholar 

  • Zhang, X., Meng, Y., de Pablos, P. O., & Sun, Y. (2019). Learning analytics in collaborative learning supported by Slack: From the perspective of engagement. Computers in Human Behavior, 92, 625–633. https://doi.org/10.1016/j.chb.2017.08.012

    Article  Google Scholar 

  • Zheng, J., Xing, W., Zhu, G., Chen, G., Zhao, H., & Xie, C. (2020). Profiling self-regulation behaviors in STEM learning of engineering design. Computers & Education, 143, 103669. https://doi.org/10.1016/j.compedu.2019.103669

  • Zhu, G., Xing, W., & Popov, V. (2019). Uncovering the sequential patterns in transformative and non-transformative discourse during collaborative inquiry learning. The Internet and Higher Education, 41, 51–61. https://doi.org/10.1016/j.iheduc.2019.02.001

  • Zimmerman, B. J. (1990). Self-regulated learning and academic achievement: An overview. Educational Psychologist, 25, 3–17. https://doi.org/10.1207/s15326985ep2501_2

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Teaching and Learning, University of Florida, Gainesville, FL, 32611, USA

    Wanli Xing

  2. National Institute of Education (NIE), Nanyang Technological University, Singapore, 637616, Singapore

    Gaoxia Zhu

  3. Department of Educational Psychology and Leadership, Texas Tech University, Lubbock, TX, 79409, USA

    Okan Arslan & Jaesub Shim

  4. Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, MI, 48109, USA

    Vitaliy Popov

Authors
  1. Wanli Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gaoxia Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Okan Arslan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jaesub Shim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vitaliy Popov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanli Xing.

Ethics declarations

Conflict of interest

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xing, W., Zhu, G., Arslan, O. et al. Using learning analytics to explore the multifaceted engagement in collaborative learning. J Comput High Educ 35, 633–662 (2023). https://doi.org/10.1007/s12528-022-09343-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12528-022-09343-0

Keywords

Search

Navigation