Authors:
Kees van Overveld
and
Tom Verhoeff
Affiliation:
Eindhoven University of Technology, Netherlands
Keyword(s):
Large Scale Assessment, Peer Reviewing, Ranking Algorithm.
Related
Ontology
Subjects/Areas/Topics:
Assessment Software Tools
;
Computer-Aided Assessment
;
Computer-Supported Education
;
Learning/Teaching Methodologies and Assessment
Abstract:
Assessing large populations of students puts a serious burden on teaching staff capacity.
For open-format assignments, automation of the reviewing process can offer only limited support.
Peer ranking is a partial solution to the problem,
with the added benefit that students' critical reading skills are developed.
We see two remaining problems, however:
(1)~for students, it is a major challenge to assign marks on an absolute scale, and
(2)~students' competence in reviewing may vary significantly---
so not all peer reviews should have a similar weight in the process.
To remedy these shortcomings, we suggest an approach to peer ranking,
inspired by Jon Kleinberg's HITS-algorithm,
where both the students' assignment results and
the quality of their double anonymous peer reviews are algorithmically ranked.
Based on preliminary model calculations,
we estimate that this strategy may reduce the required effort for reviewing open-format assignments
approximately by a factor of ten.
A first l
arge-scale pilot with this method will take place in undergraduate courses at Eindhoven University of Technology, spring 2013.
Since this involves about 900 students, automated support is a must.
We describe the peer reviewing facilities that were introduced
in our web-based education support system named peach3.
(More)