Abstract
Learning analytics (LA) and group awareness tools are regarded as top priorities for research in the field of computer-supported collaborative learning. As such, this study investigated whether LA-enabled group awareness information facilitates wiki-supported collaborative learning in secondary education. We proposed an analytic framework of measures for assessing collaboration quality in a wiki-based collaborative learning environment, covering student contribution, participation, transactivity, and social dynamics. Based on this framework, we designed an LA-enabled group awareness tool, Wikiglass, for use by both teachers and students in K-12 schools for visualizing statistics of students’ input and interactions on wikis at the class, group, and individual levels. Adopting a naturalistic design, this study allowed teachers and students to decide whether and how often to use the tool. System logs from wikis and Wikiglass and interview data were collected from 440 students and six teachers involved in semester-long wiki-supported group inquiry projects in a secondary school. Regression analyses of quantitative data and thematic content analysis of interview responses showed relationships between the frequencies of teachers’ and students’ use of Wikiglass and measures of students’ collaboration quality at both the individual and group levels. These results indicate that teachers’ scaffolding, students’ collaboration styles, and ethical issues must all be considered when implementing collaborative learning approaches for secondary education. We also discuss the implications of our results for research and practice in the application of LA and group awareness tools for enhancing wiki-supported collaborative learning in K-12 education.
Similar content being viewed by others
Availability of data and material
Instruments are available in Appendices
Code availability
Available upon request.
References
Balderas, A., Palomo-Duarte, M., Dodero, J. M., Ibarra-Sáiz, M. S., & Rodríguez-Gómez, G. (2018). Scalable authentic assessment of collaborative work assignments in wikis. International Journal of Educational Technology in Higher Education, 15(1), 40. https://doi.org/10.1186/s41239-018-0122-1
Beerwinkle, A. L. (2021). The use of learning analytics and the potential risk of harm for K-12 students participating in digital learning environments. Educational Technology Research and Development, 69(1), 327–330. https://doi.org/10.1007/s11423-020-09854-6
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57(1), 289–300. https://doi.org/10.1111/j.2517-6161.1995.tb02031.x
Bodemer, D., Janssen, J., & Schnaubert, L. (2018). Group awareness tools for computer-supported collaborative learning. International Handbook of the Learning Sciences (pp. 351–358). Routledge
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101
Bruno, E., Alexandre, B., Ferreira Mello, R., Falcão, T. P., Vesin, B., & Gašević, D. (2021). Applications of learning analytics in high schools: a Systematic Literature review. Frontiers in Artificial Intelligence, 4, 737891. https://doi.org/10.3389/frai.2021.737891
Cen, L., Ruta, D., Powell, L., Hirsch, B., & Ng, J. (2016). Quantitative approach to collaborative learning: performance prediction, individual assessment, and group composition. International Journal of Computer-Supported Collaborative Learning, 11(2), 187–225. https://doi.org/10.1007/s11412-016-9234-6
Chen, S., Ouyang, F., & Jiao, P. (2022). Promoting student engagement in online collaborative writing through a student-facing social learning analytics tool. Journal of Computer Assisted Learning, 38(1), 192–208. https://doi.org/10.1111/jcal.12604
Chounta, I. A., Bardone, E., Raudsep, A., & Pedaste, M. (2021). Exploring teachers’ perceptions of Artificial Intelligence as a tool to support their practice in Estonian K-12 education. International Journal of Artificial Intelligence in Education, 1–31. https://doi.org/10.1007/s40593-021-00243-5
Chu, S. K. W., Siu, F., Liang, M., Capio, C. M., & Wu, W. W. (2013). Users’ experiences and perceptions on using two wiki platforms for collaborative learning and knowledge management. Online Information Review, 37(2), 304–325. https://doi.org/10.1108/OIR-03-2011-0043
Cress, U., & Kimmerle, J. (2018). Collective knowledge construction. International handbook of the learning sciences, 137–146
Cress, U., Rosé, C. P., Law, N., & Ludvigsen, S. (2019). Investigating the complexity of computer-supported collaborative learning in action. International Journal of Computer-Supported Collaborative Learning, 14(2), 137–142. https://doi.org/10.1007/s11412-019-09305-2
Di Mitri, D., Schneider, J., Specht, M., & Drachsler, H. (2018). From signals to knowledge: A conceptual model for multimodal learning analytics. Journal of Computer Assisted Learning, 34(4), 338–349. https://doi.org/10.1111/jcal.12288
Everton, S. F. (2012). Disrupting dark networks. Cambridge University Press
Ferguson, C. J. (2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40(5), 532. https://doi.org/10.1037/a0015808
Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5–6), 304–317
Ferreira-Mello, R., André, M., Pinheiro, A., Costa, E., & Romero, C. (2019). Text mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(6), e1332. https://doi.org/10.1002/widm.1332
Freeman, L. C. (1978). Centrality in social networks conceptual clarification. Social networks, 1(3), 215–239
Gareth, J., Daniela, W., Trevor, H., & Robert, T. (2013). An introduction to statistical learning: with applications in R. Springer
Ghazal, S., Al-Samarraie, H., & Wright, B. (2019). A conceptualization of factors affecting collaborative knowledge building in online environments. Online Information Review, 44(1), 62–89. https://doi.org/10.1108/OIR-02-2019-0046
Hernández-García, Á., Acquila-Natale, E., Chaparro-Peláez, J., & Conde, M. (2018). Predicting teamwork group assessment using log data-based learning analytics. Computers in Human Behavior, 89, 373–384. https://doi.org/10.1016/j.chb.2018.07.016
Hmelo-Silver, C. E., & Jeong, H. (2021). An overview of CSCL methods. International Handbook of Computer-Supported Collaborative Learning, 65–83. https://doi.org/10.1007/978-3-030-65291-3_4
Hsieh, Y. C., & Hill, C. (2021). Reconceptualizing the value of peer and instructor feedback using a sequential structure. Assessment & Evaluation in Higher Education, 1–14. https://doi.org/10.1080/02602938.2021.1998341
Hu, X., Ip, J., Sadaful, K., Lui, G., & Chu, S. (2016a). Wikiglass: A learning analytic tool for visualizing collaborative wikis of secondary school students. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, 550–551. https://doi.org/10.1145/2883851.2883966
Hu, X., Ng, T. D. J., Tian, L., & Lei, C. U. (2016b). Automating assessment of collaborative writing quality in multiple stages: The case of wiki. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, 518–519. https://doi.org/10.1145/2883851.2883963
Hu, X. (2017). Automated recognition of thinking orders in secondary school student writings. Learning: Research and Practice, 3(1), 30–41. https://doi.org/10.1080/23735082.2017.1284253
Hu, X., Yang, C., Qiao, C., Lu, X., & Chu, S. K. (2017). New features in Wikiglass, a learning analytic tool for visualizing collaborative work on wikis. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference. 616–617. https://doi.org/10.1145/3027385.3029489
Hu, X., Ng, J. T. D., & Lei, C. U. (2021a). Evaluation of a lightweight learning analytics tool in Moodle and edX: Preliminary results. In Proceedings of IEEE International Conference on Advanced Learning Technologies (ICALT), 157–159. https://doi.org/10.1109/ICALT52272.2021a.00054
Hu, X., Ng, J., & Jiang, X. (2021b). Young students’ experience of analytics-supported CSCL and the influence of parental attitudes. In Proceedings of the 14th International Conference on Computer-Supported Collaborative Learning-CSCL 2021b. International Society of the Learning Sciences. https://doi.dx.org/10.22318/cscl2021b.225
Janssen, J., & Bodemer, D. (2013). Coordinated computer-supported collaborative learning: Awareness and awareness tools. Educational Psychologist, 48(1), 40–55. https://doi.org/10.1080/00461520.2012.749153
Järvelä, S., Kirschner, P. A., Hadwin, A., Järvenoja, H., Malmberg, J., Miller, M., & Laru, J. (2016). Socially shared regulation of learning in CSCL: Understanding and prompting individual-and group-level shared regulatory activities. International Journal of Computer-Supported Collaborative Learning, 11(3), 263–280. https://doi.org/10.1007/s11412-016-9238-2
Jena, A. K., Bhattacharjee, S., Gupta, S., Das, J., & Debnath, R. (2018). Exploring the effects of Web 2.0 technology on individual and collaborative learning performance in relation to self-regulation of learners. Journal on School Educational Technology, 13(4), 20–35
Kaliisa, R., Rienties, B., Mørch, A. I., & Kluge, A. (2022). Social learning analytics in computer-supported collaborative learning environments: A systematic review of empirical studies. Computers and Education Open, 100073. https://doi.org/10.1016/j.caeo.2022.100073
Kear, K., Donelan, H., & Williams, J. (2014). Using wikis for online group projects: Student and tutor perspectives. The International Review of Research in Open and Distributed Learning, 15(4), 70–90. https://doi.org/10.19173/irrodl.v15i4.1753
Kim, J., Jo, I. H., & Park, Y. (2016). Effects of learning analytics dashboard: analyzing the relations among dashboard utilization, satisfaction, and learning achievement. Asia Pacific Education Review, 17(1), 13–24. https://doi.org/10.1007/s12564-015-9403-8
Koppenborg, M., & Klingsieck, K. B. (2022). Group work and student procrastination. Learning and Individual Differences, 94, 102117. https://doi.org/10.1016/j.lindif.2022.102117
Kovanovic, V., Mazziotti, C., & Lodge, J. (2021). Learning analytics for primary and secondary schools. Journal of Learning Analytics, 8(2), 1–5. https://doi.org/10.18608/jla.2021.7543
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into practice, 41(4), 212–218. https://doi.org/10.1207/s15430421tip4104_2
Laurillard, D. (2009). The pedagogical challenges to collaborative technologies. International Journal of Computer-Supported Collaborative Learning, 4(1), 5–20
Law, N., & Liang, L. (2020). A multilevel framework and method for learning analytics integrated learning design. Journal of Learning Analytics, 7(3), 98–117. https://doi.org/10.18608/jla.2020.73.8
Ludvigsen, S., Cress, U., Law, N., Stahl, G., & Rosé, C. P. (2018). Multiple forms of regulation and coordination across levels in educational settings. International Journal of Computer-Supported Collaborative Learning, 13(1), 1–6. https://doi.org/10.1007/s11412-018-9274-1
Lyons, K. M., Lobczowski, N. G., Greene, J. A., Whitley, J., & McLaughlin, J. E. (2021). Using a design-based research approach to develop and study a web-based tool to support collaborative learning. Computers & Education, 161, 104064. https://doi.org/10.1016/j.compedu.2020.104064
Mangaroska, K., & Giannakos, M. (2018). Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies, 12(4), 516–534. https://doi.org/10.1109/TLT.2018.2868673
Margaliot, A., Gorev, D., & Vaisman, T. (2018). How student teachers describe the online collaborative learning experience and evaluate its contribution to their learning and their future work as teachers. Journal of Digital Learning in Teacher Education, 34(2), 88–102. https://doi.org/10.1080/21532974.2017.1416710
Meijer, H., Hoekstra, R., Brouwer, J., & Strijbos, J. W. (2020). Unfolding collaborative learning assessment literacy: a reflection on current assessment methods in higher education. Assessment & Evaluation in Higher Education, 45(8), 1222–1240. https://doi.org/10.1080/02602938.2020.1729696
Melzner, N., Greisel, M., Dresel, M., & Kollar, I. (2020). Regulating self-organized collaborative learning: the importance of homogeneous problem perception, immediacy and intensity of strategy use. International Journal of Computer-Supported Collaborative Learning, 15(2), 149–177. https://doi.org/10.1007/s11412-020-09323-5
Ng, J., Hu, X., Luo, M., & Chu, S. K. (2019). Relations among participation, fairness and performance in collaborative learning with Wiki-based analytics. Proceedings of the Association for Information Science and Technology, 56(1), 463–467. https://doi.org/10.1002/pra2.48
Ng, J., Lei, L., Iseli-Chan, N., Li, J., Siu, F., Chu, S., & Hu, X. (2020). Non-repository uses of learning management system through mobile access. Journal of Educational Technology Development and Exchange, 13(1), 1–20. https://doi.org/10.18785/jetde.1301.01
Ng, J. T. D., Wang, Z., & Hu, X. (2022). Needs analysis and prototype evaluation of student-facing LA dashboard for virtual reality content creation. In Proceedings of LAK22: 12th International Learning Analytics and Knowledge Conference, 444–450. https://doi.org/10.1145/3506860.3506880
Notari, M., Reynolds, R., Chu, S. K. W., & Honegger, B. D. (2016). The wiki way of learning; Creating learning experiences using collaborative web pages. Chicago: American Library Association
Ochoa, X., & Wise, A. F. (2021). Supporting the shift to digital with student-centered learning analytics. Educational Technology Research and Development, 69(1), 357–361. https://doi.org/10.1007/s11423-020-09882-2
Ollesch, L., Heimbuch, S., & Bodemer, D. (2021). Improving learning and writing outcomes: Influence of cognitive and behavioral group awareness tools in wikis. International Journal of Computer-Supported Collaborative Learning, 16, 225–259. https://doi.org/10.1007/s11412-021-09346-6
Oluwajana, D., Adeshola, I., & Clement, S. (2021). Does the use of a web-based collaborative platform reduce cognitive load and influence project-based student engagement? Current Psychology, 1–14. https://doi.org/10.1007/s12144-021-02145-0
Perrow, M. (2017). Strengthening the conversation in blended and face-to face courses: Connecting online and in-person learning with crossover protocols. College Teaching, 65(3), 97–105. https://doi.org/10.1080/87567555.2017.1300869
Prieto-Alvarez, C. G., Martinez-Maldonado, R., & Anderson, T. (2018). Co-designing learning analytics tools with learners. In J. M. Lodge, J. C. Horvath, & L. Corrin (Eds.), Learning analytics in the classroom: Translating learning analytics research for teachers. Abingdon-on-Thames, UK: Routledge
Radmehr, F., & Drake, M. (2018). An assessment-based model for exploring the solving of mathematical problems: Utilizing revised bloom’s taxonomy and facets of metacognition. Studies in Educational Evaluation, 59, 41–51. https://doi.org/10.1016/j.stueduc.2018.02.004
Saqr, M., Fors, U., Tedre, M., & Nouri, J. (2018). How social network analysis can be used to monitor online collaborative learning and guide an informed intervention. PloS One, 13(3), e0194777. https://doi.org/10.1371/journal.pone.0194777
Saqr, M., Nouri, J., Vartiainen, H., & Tedre, M. (2020a). Robustness and rich clubs in collaborative learning groups: a learning analytics study using network science. Scientific Reports, 10(1), 1–16. https://doi.org/10.1038/s41598-020-71483-z
Saqr, M., Viberg, O., & Vartiainen, H. (2020b). Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter? International Journal of Computer-Supported Collaborative Learning, 15(2), 227–248. https://doi.org/10.1007/s11412-020-09322-6
Schnaubert, L., & Bodemer, D. (2022). Group awareness and regulation in computer-supported collaborative learning. International Journal of Computer-Supported Collaborative Learning, 17(1), 11–38. https://doi.org/10.1007/s11412-022-09361-1
Schneider, B., Dowell, N., & Thompson, K. (2021). Collaboration analytics—current state and potential futures. Journal of Learning Analytics, 8(1), 1–12. https://doi.org/10.18608/jla.2021.7447
Schreurs, B., Cornelissen, F., & De Laat, M. (2019). How do online learning networks emerge? A review study of self-organizing network effects in the field of networked learning. Education Sciences, 9(4), 289. https://doi.org/10.3390/educsci9040289
Spada, H., Meier, A., Rummel, N., & Hauser, S. (2005). A new method to assess the quality of collaborative process in CSCL. In T. Koschmann (Ed.), Computer Supported Collaborative Learning 2005: The Next 10 Years! (pp. 622–631). Routledge
Stahl, G. (2017). Group practices: A new way of viewing CSCL. International Journal of Computer-Supported Collaborative Learning, 12(1), 113–126. https://doi.org/10.1007/s11412-017-9251-0
Strauß, S., & Rummel, N. (2021). Promoting regulation of equal participation in online collaboration by combining a group awareness tool and adaptive prompts. But does it even matter? International Journal of Computer-Supported Collaborative Learning, 16(1), 67–104. https://doi.org/10.1007/s11412-021-09340-y
Strijbos, J. W. (2016). Assessment of collaborative learning. Brown & Harris (Eds), Handbook of Human and Social Conditions in Assessment (pp. 302–318). Routledge
Su, Y., Li, Y., Hu, H., & Rosé, C. P. (2018). Exploring college English language learners’ self and social regulation of learning during wiki-supported collaborative reading activities. International Journal of Computer-Supported Collaborative Learning, 13(1), 35–60. https://doi.org/10.1007/s11412-018-9269-y
Tchounikine, P. (2019). Learners’ agency and CSCL technologies: towards an emancipatory perspective. International Journal of Computer-Supported Collaborative Learning, 14(2), 237–250. https://doi.org/10.1007/s11412-019-09302-5
van Aalst, J. (2013). Assessment in collaborative learning. Hmelo-Silver, Chinn, Chan, & O’Donnell (Eds). The International Handbook of Collaborative Learning (pp. 280–296). Routledge
van Leeuwen, A., Rummel, N., & Van Gog, T. (2019). What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations? International Journal of Computer-Supported Collaborative Learning, 14(3), 261–289. https://doi.org/10.1007/s11412-019-09299-x
Verdú, M. J., De Castro, J. P., Regueras, L. M., & Corell, A. (2021). MSocial: Practical integration of social learning analytics into Moodle. Ieee Access : Practical Innovations, Open Solutions, 9, 23705–23716. https://doi.org/10.1109/ACCESS.2021.3056914
Vieira, C., Parsons, P., & Byrd, V. (2018). Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education, 122, 119–135. https://doi.org/10.1016/j.compedu.2018.03.018
Winstone, N., & Carless, D. (2019). Designing Effective Feedback Processes in Higher Education: A Learning-focused Approach. London: Routledge
Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467. https://doi.org/10.1007/s11412-017-9267-5
Xie, K., Di Tosto, G., Lu, L., & Cho, Y. S. (2018). Detecting leadership in peer-moderated online collaborative learning through text mining and social network analysis. The Internet and Higher Education, 38, 9–17. https://doi.org/10.1016/j.iheduc.2018.04.002
Xing, W., Wadholm, R., Petakovic, E., & Goggins, S. (2015). Group learning assessment: Developing theory-informed analytics. Journal of Educational Technology & Society, 18(2), 110–128
Zhang, X., Meng, Y., de Pablos, P. O., & Sun, Y. (2017). Learning analytics in collaborative learning supported by Slack: From the perspective of engagement. Computers in Human Behavior, 92, 625–633. https://doi.org/10.1016/j.chb.2017.08.012
Zheng, B., Niiya, M., & Warschauer, M. (2015). Wikis and collaborative learning in higher education. Technology Pedagogy and Education, 24(3), 357–374. https://doi.org/10.1080/1475939X.2014.948041
Acknowledgements
This study is supported by Research Grants Council of the Hong Kong S. A. R., China. (No. HKU 27401114). The authors would like to thank the students and teachers who participated in this study. Prof. Carol K. K. Chan, Dr. Jan van Aalst, and the anonymous reviewers are highly appreciated for their valuable comments and suggestions on earlier drafts of this article.
Funding
This work was supported by the Research Grants Council, University Grants Committee, Hong Kong [HKU 27401114].
Author information
Authors and Affiliations
Contributions
XH: Conceptualization, Methodology, Literature review, Theoretical framework, Supervision, Data analysis, Result interpretation; Manuscript writing and review; JN: Literature review, Data collection, Data analysis; Manuscript writing; SC: Conceptualization, Methodology, Data collection, Result interpretation, Manuscript review.
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests
Not applicable.
Ethics approval
Approved by the researchers’ institution.
Consent to participate
All participants signed informed consents
Consent for publication
All authors consent for publication
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix I: Teacher interview protocol
-
1.
How did you assess students’ collaboration in the group project?
-
2.
Why and how was Wikiglass (not) helpful for you in the group project?
Appendix II: Student interview protocol
-
1.
Did you plan and organize your own work in the group project? If yes, how? If no, why?
-
2.
Did you coordinate the work with groupmates in the group project? If yes, how? If no, why?
-
3.
How did you feel the quality of collaboration in your group?
-
4.
How was work distributed in your group? Was it fair? Why (not)?
-
5.
Why and how was Wikiglass (not) useful in the group project?
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hu, X., Ng, J.T.D. & Chu, S.K.W. Implementing learning analytics in wiki-supported collaborative learning in secondary education: A framework-motivated empirical study. Intern. J. Comput.-Support. Collab. Learn 17, 427–455 (2022). https://doi.org/10.1007/s11412-022-09377-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11412-022-09377-7