Welcome to the InfoSci Platform
Incentivizing High-Quality Reviews in Peer-to-Peer Settings: A Feasibility Study with Student Assignments

Incentivizing High-Quality Reviews in Peer-to-Peer Settings: A Feasibility Study with Student Assignments

J.Z. Yue, K. Böhm, S. von Stackelberg
Copyright: © 2014 |Volume: 6 |Issue: 1 |Pages: 21
ISSN: 1942-9010|EISSN: 1942-9029|EISBN13: 9781466657243|DOI: 10.4018/ijvcsn.2014010101
Cite Article Cite Article

MLA

Yue, J.Z., et al. "Incentivizing High-Quality Reviews in Peer-to-Peer Settings: A Feasibility Study with Student Assignments." IJVCSN vol.6, no.1 2014: pp.1-21. http://doi.org/10.4018/ijvcsn.2014010101

APA

Yue, J., Böhm, K., & von Stackelberg, S. (2014). Incentivizing High-Quality Reviews in Peer-to-Peer Settings: A Feasibility Study with Student Assignments. International Journal of Virtual Communities and Social Networking (IJVCSN), 6(1), 1-21. http://doi.org/10.4018/ijvcsn.2014010101

Chicago

Yue, J.Z., K. Böhm, and S. von Stackelberg. "Incentivizing High-Quality Reviews in Peer-to-Peer Settings: A Feasibility Study with Student Assignments," International Journal of Virtual Communities and Social Networking (IJVCSN) 6, no.1: 1-21. http://doi.org/10.4018/ijvcsn.2014010101

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

Peer reviewing has been touted as a popular instrument to identify good contributions in communities. A problem of peer reviewing is that reviewers have little incentive to make significant effort. To address this problem, the authors introduce a new variant of peer reviewing. It differs from conventional peer reviewing in two ways: First, peers who have made a contribution must also review the contributions made by others. Second, each contributor issues ratings regarding the reviews he has received. To incentivize reviewing, they design an assessment scheme which does not only assess the quality of the contribution made by a peer, but also the quality of the reviews he has submitted. The scheme ranks peers by overall performance, and the ranks determine their payoff. Such a setting gives way to competition among peers. A core challenge however is to elicit objective reviews and ratings. The authors consider two issues which are in the way of this objectiveness: First, they expect preference bias in ratings, i.e., peers tend to prefer reviews with high scores, but dislike reviews with low scores. Second, strategic peers might defame others in their reviews or ratings. This is because they perceive others as competitors. In this paper, they propose a heuristic to address these issues. Further, they carry out a user study in a lecture scenario to evaluate their scheme. It shows that students are incentivized to submit high-quality reviews and that their scheme is effective to evaluate the performance of students.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.