Abstract
Evaluation plays a substantial role in group work implementation and peer evaluation gets prevalent with increasing flipped learning scenarios and online evaluation platforms. The accuracy of peer evaluation remains contingent in group work practice thus eliciting relevant studies on grader reliability. In this study, we present a data-driven approach to solving this issue utilizing learner models. On the one hand, we use previous learning logs to estimate and visualize the grader reliability in group work evaluation sessions as “peer evaluation potential”, which is used to align peer rating accuracy. On the other hand, leveraging reliability indicators created in the current session, learner models can be updated with new dimensions for subsequent usage. In addition, a case study in a high school English class was presented to examine this data-driven workflow and the results suggest the estimated peer evaluation potential correlates with the deviation from average peer judgment. Further potentials to cultivate peer evaluation-related capabilities are proposed as well.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abou-Khalil, V., Ogata, H.: Homogeneous student engagement: a strategy for group formation during online learning. In: Hernández-Leo, D., Hishiyama, R., Zurita, G., Weyers, B., Nolte, A., Ogata, H. (eds.) CollabTech 2021. LNCS, vol. 12856, pp. 85–92. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85071-5_6
Aminu, N., Hamdan, M., Russell, C.: Accuracy of self-evaluation in a peer-learning environment: an analysis of a group learning model. SN Soc. Sci. 1(7), 1–17 (2021)
Bjelobaba, G., Paunovic, M., Savic, A., Stefanovic, H., Doganjic, J., Miladinovic Bogavac, Z.: Blockchain technologies and digitalization in function of student work evaluation. Sustainability 14(9), 5333 (2022)
Boticki, I., Akçapınar, G., Ogata, H.: E-book user modelling through learning analytics: the case of learner engagement and reading styles. Interact. Learn. Environ. 27(5–6), 754–765 (2019)
Bozic, N.H., Mornar, V., Boticki, I.: Introducing adaptivity and collaborative support into a web-based LMS. Comput. Inform. 27(4), 639–659 (2008)
Forsell, J., Forslund Frykedal, K., Hammar Chiriac, E.: Group work assessment: assessing social skills at group level. Small Group Res. 51(1), 87–124 (2020)
Kasch, J., van Rosmalen, P., Löhr, A., Klemke, R., Antonaci, A., Kalz, M.: Students’ perceptions of the peer-feedback experience in MOOCs. Distance Educ. 42(1), 145–163 (2021)
Liang, C., Majumdar, R., Ogata, H.: Learning log-based automatic group formation: system design and classroom implementation study. Res. Pract. Technol. Enhanced Learn. 16(1), 1–22 (2021). https://doi.org/10.1186/s41039-021-00156-w
Changhao, L., Toyokawa, Y., Nakanishi, T., Majumdar, R., Ogata, H.: Supporting peer evaluation in a data-driven group learning environment. In: Hernández-Leo, D., Hishiyama, R., Zurita, G., Weyers, B., Nolte, A., Ogata, H. (eds.) CollabTech 2021. LNCS, vol. 12856, pp. 93–100. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85071-5_7
Masaki, U., Maomi, U., et al.: Item response theory with assessors’ parameters of peer assessment. J. Inst. Electron. Inf. Commun. Eng. 91(2), 377–388 (2008)
Piech, C., Huang, J., Chen, Z., Do, C., Ng, A., Koller, D.: Tuned models of peer assessment in MOOCs. arXiv preprint arXiv:1307.2579 (2013)
Pond, K., Ul-Haq, R.: Learning to assess students using peer review. Stud. Educ. Eval. 24, 331–348 (1997)
Strijbos, J.W.: Assessment of (computer-supported) collaborative learning. IEEE Trans. Learn. Technol. 4(1), 59–73 (2010)
Ismail, N., et al.: Peer evaluation system in team work skills assessment. In: Fook, C.Y., Sidhu, G.K., Narasuman, S., Fong, L.L., Abdul Rahman, S.B. (eds.) 7th International Conference on University Learning and Teaching (InCULT 2014) Proceedings, pp. 603–616. Springer, Singapore (2016). https://doi.org/10.1007/978-981-287-664-5_47
Willey, K., Gardner, A.: Investigating the capacity of self and peer assessment activities to engage students and promote learning. Eur. J. Eng. Educ. 35(4), 429–443 (2010)
Acknowledgements
This research was supported by JSPS KAKENHI 20K20131, 20H01722, 22H03902, NEDO JPNP18013, JPNP20006, and JST SPRING JPMJSP2110.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, C., Gorham, T., Horikoshi, I., Majumdar, R., Ogata, H. (2022). Estimating Peer Evaluation Potential by Utilizing Learner Model During Group Work. In: Wong, LH., Hayashi, Y., Collazos, C.A., Alvarez, C., Zurita, G., Baloian, N. (eds) Collaboration Technologies and Social Computing. CollabTech 2022. Lecture Notes in Computer Science, vol 13632. Springer, Cham. https://doi.org/10.1007/978-3-031-20218-6_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-20218-6_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20217-9
Online ISBN: 978-3-031-20218-6
eBook Packages: Computer ScienceComputer Science (R0)