Abstract
Peer assessment has an important role in teaching and learning nowadays. However, the existing techniques tend to be limited due to a lack of a suitable means to evaluate the accuracy of peer assessment activities and supervise peer assessment performances in an online environment. Hence, it is very necessary to develop new solutions encourage students to take full effort when they join in online peer assessment activities. This study proposed a new approach to evaluate the accuracy of peer assessment results for enhancing the quality of peer assessment system. The proposed approach was applied to evaluate peer assessment activities of two groups of students. The experimental results analyzed and evaluated in terms of both the learning scores and opinion survey results, which revealed that when students realized both their own responses and their peer assessment activities were taken into account by the system in the calculation of the last score, they had positive attitudes towards peer assessment activities and the peer assessment results were improved significantly.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Adiguzel, T., Varank, I., Erkoc, M. F., & Buyukimdat, M. K. (2017). Examining a web-based peer feedback system in an introductory computer literacy course. Eurasia Journal of Mathematics, Science and Technology Education, 13(1), 237–251. https://doi.org/10.12973/eurasia.2017.00614a
AL-Smadi, M., Gütl, C., & Helic, D. (2009). Using computers to enhance peer-assessment activities: Towards a flexible e-assessment system. International Conference Interactive Computer Aided Learning, Villach, Austria
Ashenafi, M. M., Ronchetti, M., & Riccardi, G. (2017). Exploring the role of online peer-assessment as a tool of early intervention. Emerging Technologies for Education, Springer, Cham.
Bouzidi, L., & Jaillet, A. (2009). Can online peer assessment be trusted? Journal of Educational Technology and Society, 12(4), 257–268. https:https://doi.org/www.jstor.org/stable/jeductechsoci.12.4.257
Chang, T. Y., & Chen, Y. T. (2009). Cooperative learning in E-learning: A peer assessment of student-centered using consistent fuzzy preference. Expert Systems with Applications, 36(2009), 8342–8349. https://doi.org/10.1016/j.eswa.2008.10.050
Chien, S. Y., Hwang, G. J., & Jong, M. S. Y. (2019). Effects of peer assessment within the context of spherical video-based virtual reality on EFL students’ Englishspeaking performance and learning perceptions. Computers and Education. https://doi.org/10.1016/j.compedu.2019.103751
Devi, M. S., & Mittal, H. (2016). Machine learning techniques with ontology for subjective answer evaluation. International Journal on Natural Language Computing (IJNLC), 5(2), 1–11. https://doi.org/10.5121/ijnlc.2016.5201
Foo, S. (2021). Analysing peer feedback in asynchronous online discussions: A case study. Education and Information Technologies, 26(2021), 4553–4572. https://doi.org/10.1007/s10639-021-10477-4
Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20, 304–315. https://doi.org/10.1016/j.learninstruc.2009.08.007
Ha, M., Nehm, R. H., Urban-Lurain, M., & Merrill, J. E. (2011). Applying computerized-scoring models of written biological explanations across courses and colleges: Prospects and limitations. CBE - Life Sciences Education. https://doi.org/10.1187/cbe.11-08-0081
Haudek, K. C., Kaplan, J. J., Knight, J., Long, T., Merrill, J., Munn, A., Nehm, R. H., Smith, M., & Urban-Lurain, M. (2011). Harnessing technology to improve formative assessment of student conceptions in STEM: Forging a national network. CBE-Life Sciences Education, 10(Summer), 149–155. https://doi.org/10.1187/cbe.11-03-0019
Hoang, L. P., Ngamnij, A., Somjit, A., & Wararat, R. (2012). Multi-dimensional assessment on free-text answers to enhance leaners’ activities and collaborations. (Ed.),^(Eds.). Proceedings of the 2012 IEEE 16th International Conference on CSCWD, Wuhan, China.
Hoang, L. P., Phuong, L. A., Arch-int, S., & Arch-Int, N. (2020). Multidimensional assessment of open-ended questions for enhancing the quality of peer assessment in E-learning environments. In: P. Hershey (Ed.), Management Association (Ed.), Learning and Performance Assessment: Concepts, Methodologies, Tools, and Applications (Vol. 1, pp. 147–173). IGI Global. https://doi.org/10.4018/978-1-7998-0420-8.ch008
Hou, W. J., Tsao, J. H., Li, S. Y., & Chen, L. (2010). Automatic assessment of students’ free-text answers with support vector machines. Trends in Applied Intelligent Systems, Berlin, Heidelberg.
Hsia, L. H., & Sung, H. Y. (2020). Effects of a mobile technology-supported peer assessment approach on students’ learning motivation and perceptions in a college flipped dance class. International Journal of Mobile Learning and Organisation, 14(1), 99–113. https://doi.org/10.1504/IJMLO.2020.103892
Kaufman, J. H., & Schunn, C. D. (2011). Students’ perceptions about peer assessment for writing: Their origin and impact on revision work. Instructional Science, 39(2011), 387–406. https://doi.org/10.1007/s11251-010-9133-6
Kilic, G. B., & Cakan, M. (2017). Peer assessment of elementary science teaching skills. Journal of Science Teacher Education, 18(1), 91–107. https://doi.org/10.1007/s10972-006-9021-8
Kollar, I., & Fischer, F. (2010). Peer assessment as collaborative learning: A cognitive perspective. Learning and Instruction, 20, 344–348. https://doi.org/10.1016/j.learninstruc.2009.08.005
Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2019). Does peer assessment promote student learning? A meta-analysis. Assessment and Evaluation in Higher Education, 45(2), 193–211. https://doi.org/10.1080/02602938.2019.1620679
Liu, J., Guo, X., Gao, R., Fram, P., Ling, Y., Zhang, H., & Wang, J. (2019). Students’ learning outcomes and peer rating accuracy in compulsory and voluntary online peer assessment. Assessment and Evaluation in Higher Education, 44(6), 835–847. https://doi.org/10.1080/02602938.2018.1542659
López-Pellisa, T., Rotger, N., & Rodríguez-Gallego, F. (2021). Collaborative writing at work: Peer feedback in a blended learning environment. Education and Information Technologies, 26(2021), 1293–1310. https://doi.org/10.1007/s10639-020-10312-2
Lu, J., & Law, N. (2012). Online peer assessment: Effects of cognitive and affective feedback. Instructional Science. https://doi.org/10.1007/s11251-011-9177-2
Majdoddin, K. (2010). Peer assessment: An alternative to traditional testing. The Modern Journal of Applied Linguistics, 2(2), 396–405.
Nehm, R. H., & Ha, M. (2010). Item feature effects in evolution assessment. Journal of Research in Science Teaching, 48(3), 237–256. https://doi.org/10.1002/tea.20400
Nehm, R. H., & Haertig, H. (2012). Human vs. computer diagnosis of students’ natural selection knowledge: Testing the efficacy of text analytic software. Journal of Science Education and Technology, 21, 56–73. https://doi.org/10.1007/s10956-011-9282-7
Nehm, R. H., Ha, M., & Mayfield, E. (2012). Transforming Biology assessment with machine learning: Automated scoring of written evolutionary explanations. Journal of Science Education and Technology, 21, 183–196. https://doi.org/10.1007/s10956-011-9300-9
Noorbehbahani, F., & Kardan, A. A. (2011). The automatic assessment of free text answers using a modified BLEU algorithm. Computers and Education, 56, 337–345. https://doi.org/10.1016/j.compedu.2010.07.013
Papinczak, T., Young, L., & Groves, M. (2007). Peer assessment in problem-based learning: A qualitative study. Advances in Health Sciences Education, 21, 169–186. https://doi.org/10.1007/s10459-005-5046-6
Potgieter, C. (2012). Linking learning activities and assessment activities to learning outcomes and assessment standards when teaching technology: A case study. International Journal of Technology and Design Education, 23(2013), 969–986. https://doi.org/10.1007/s10798-012-9226-4
Rada, R., Michailidis, A., & Wang, W. (1994). Collaborative hypermedia in a classroom setting. Journal of Educational Multimedia and Hypermedia, 3(1), 21–36.
Reinholz, D. (2016). The assessment cycle: A model for learning through peer assessment. Assessment and Evaluation in Higher Education, 41(2), 301–315. https://doi.org/10.1080/02602938.2015.1008982
Seery, N., Canty, D., & Phelan, P. (2012). The validity and value of peer assessment using adaptive comparative judgement in design driven practical education. International Journal of Technology and Design Education, 22(2012), 205–226. https://doi.org/10.1007/s10798-011-9194-0
Seifert, T., & Feliks, O. (2019). Online self-assessment and peer-assessment as a tool to enhance student-teachers’ assessment skills. Assessment and Evaluation in Higher Education, 44(2), 169–185. https://doi.org/10.1080/02602938.2018.1487023
Strijbos, J. W., & Sluijsmans, D. (2010). Unravelling peer assessment: Methodological, functional, and conceptual developments. Learning and Instruction, 20, 265–269. https://doi.org/10.1016/j.learninstruc.2009.08.002
Sun, D., & Looi, C. K. (2013). Designing a web-based science learning environment for model-based collaborative inquiry. Journal of Science Education and Technology, 22(2013), 73–89. https://doi.org/10.1007/s10956-012-9377-9
Tillema, H., Leenknecht, M., & Segers, M. (2011). Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning – A review of research studies. Studies in Educational Evaluation, 37(1), 25–34. https://doi.org/10.1016/j.stueduc.2011.03.004
Topping, K. J. (2010). Methodological quandaries in studying process and outcomes in peer assessment. Learning and Instruction, 20(4), 339–343. https://doi.org/10.1016/j.learninstruc.2009.08.003
Tran, H. V., Phan, T. C., Le, H. T., Hoang, L. P., Phan, T. M., & Thai, H. T. (2021). A systematic style-based blended teaching for competence enhancement of lecturers in the COVID-19 pandemic situation: A case study for teaching in higher education. Turkish Journal of Computer and Mathematics Education. https://doi.org/10.16949/turkbilmat.702540
Tsai, C. C., & Liang, J. C. (2009). The development of science activities via on-line peer assessment: The role of scientific epistemological views. Instructional Science. https://doi.org/10.1007/s11251-007-9047-0
Turcotte, S. (2012). Computer-supported collaborative inquiry on buoyancy: A discourse analysis supporting the ‘“Pieces”’ Position on conceptual change. Journal of Science Education and Technology, 21(2012), 808–825. https://doi.org/10.1007/s10956-012-9368-x
Wang, C., Ke, Y. T., Wu, J. T., & Hsu, W. H. (2012). Collaborative action research on technology integration for science learning. Journal of Science Education and Technology, 21(2012), 125–132. https://doi.org/10.1007/s10956-011-9289-0
Wen, M. L., & Tsai, C. C. (2006). University students’ perceptions of and attitudes toward (online) peer assessment. Higher Education, 51(2006), 27–44. https://doi.org/10.1007/s10734-004-6375-8
Zheng, L., Zhang, X., & Cui, P. (2020). The role of technology-facilitated peer assessment and supporting strategies: A meta-analysis. Assessment and Evaluation in Higher Education, 45(3), 372–386. https://doi.org/10.1080/02602938.2019.1644603
Acknowledgements
This research is funded by Ministry of Education and Training of Viet Nam under project number B2021-DHH-11.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Hoang, L.P., Le, H.T., Van Tran, H. et al. Does evaluating peer assessment accuracy and taking it into account in calculating assessor’s final score enhance online peer assessment quality?. Educ Inf Technol 27, 4007–4035 (2022). https://doi.org/10.1007/s10639-021-10763-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10639-021-10763-1