Yanxin Lu
ABSTRACT
In this paper, we detail our efforts at creating and running a controlled study designed to examine how students in a MOOC might be motivated to do a better job during peer grading. This study involves more than one thousand students of a popular MOOC. We ask two specific questions: (1) When a student knows that his or her own peer grading efforts are being examined by peers, does this knowledge alone tend to motivate the student to do a better job when grading assignments? And (2) when a student not only knows that his or her own peer grading efforts are being examined by peers, but he or she is also given a number of other peer grading efforts to evaluate (so the peer graders see how other peer graders evaluate assignments), do both of these together tend to motivate the student to do a better job when grading assignments? We find strong statistical evidence that ``grading the graders'' does in fact tend to increase the quality of peer grading.
- Susan Adams. Is coursera the beginning of the end for traditional higher education? Higher Education, 2012.Google Scholar
- Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. Steering User Behavior with Badges. In Proceedings of WWW, pages 95--106, 2013. Google ScholarDigital Library
- Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. Engaging with Massive Online Courses. In Proceedings of WWW, pages 687--698, 2014. Google ScholarDigital Library
- George EP Box, William Gordon Hunter, J Stuart Hunter, et al. Statistics for experimenters. John Wiley and sons New York, 1978.Google Scholar
- Lori B. Breslow, David E. Pritchard, Jennifer DeBoer, Glenda S. Stump, Andrew D. Ho, and Daniel T. Seaton. Studying learning in the worldwide classroom: Research into edX's first mooc. Research & Practice in Assessment, 8:13--25, 2013.Google Scholar
- D. Casey, E. Burke, C. Houghton, L. Mee, R. Smith, D. Van Der Putten, H. Bradley, and M. Folan. Use of peer assessment as a student engagement strategy in nurse education. Nursing & Health Sciences, 13(4):514--520, 2011.Google ScholarCross Ref
- Jo Davies and Martin Graff. Performance in e-learning: online participation and student grades. British Journal of Educational Technology, 36(4):657--663, July 2005.Google ScholarCross Ref
- J.P. Dineen, H.B. Clark, and T.R. Risley. Peer tutoring among elementary students: Educational benefits to the tutor. Journal of Applied Behavior Analysis, 10(2):231, 1977.Google ScholarCross Ref
- A. DiPardo and S.W. Freedman. Peer response groups in the writing classroom: Theoretic foundations and new directions. Review of Educational Research, 58(2):119--149, 1988.Google ScholarCross Ref
- F. Dochy, M. Segers, and D. Sluijsmans. The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education, 24(3):331--350, 1999.Google ScholarCross Ref
- J. Dunlosky, K. A. Rawson, E. J. Marsh, M. J. Nathan, and D. T. Willingham. Improving students' learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 2012.Google Scholar
- David Easley and Arpita Ghosh. Incentives, Gamification, and Game Theory: An Economic Approach to Badge Design. In Proceedings of the Fourteenth ACM Conference on Electronic Commerce, EC '13, pages 359--376, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- Bradley Efron and B Efron. The jackknife, the bootstrap and other resampling plans, volume 38. SIAM, 1982.Google Scholar
- P.A. Ertmer, J.C. Richardson, B. Belland, D. Camin, P. Connolly, G. Coulthard, K. Lei, and C. Mong. Using peer feedback to enhance the quality of student online postings: An exploratory study. Journal of Computer-Mediated Communication, 12(2):412--433, 2007.Google ScholarCross Ref
- N. Falchikov and J. Goldfinch. Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research, 70(3):287--322, 2000.Google Scholar
- Nancy Falchikov. Peer feedback marking: Developing peer assessment. Innovations in Education and Training International, 32(2):175--187, 1995.Google ScholarCross Ref
- Lisa E. Gueldenzoph and Gary L. May. Collaborative peer evaluation: Best practices for group member assessments. Business Communication Quarterly, 65(1):9--20, 2002.Google ScholarCross Ref
- Khe Foon Hew and Wing Sum Cheung. Students and instructors use of massive open online courses (moocs): Motivations and challenges. Educational Research Review, 12(0):45 -- 58, 2014.Google ScholarCross Ref
- James A Keaten and M. Elizabeth Richardson. A field investigation of peer assessment as part of the student group grading process, 1993.Google Scholar
- René F. Kizilcec, Chris Piech, and Emily Schneider. Deconstructing Disengagement: Analyzing Learner Subpopulations in Massive Open Online Courses. In Proceedings of the International Conference on Learning Analytics and Knowledge, LAK '13, pages 170--179. ACM, 2013. Google ScholarDigital Library
- Chinmay Kulkarni, Koh Pang Wei, Huy Le, Daniel Chia, Kathryn Papadopoulos, Justin Cheng, Daphne Koller, and Scott R. Klemmer. Peer and self assessment in massive online classes. ACM Trans. Comput.-Hum. Interact., 20(6):33:1--33:31, December 2013. Google ScholarDigital Library
- Y. Lai. Which do students prefer to evaluate their essays: Peers or computer program. British Journal of Educational Technology, 41(3):432--454, 2009.Google ScholarCross Ref
- Rupert G Miller. Simultaneous statistical inference, volume 196. Springer, 1966.Google Scholar
- Laura Pappano. The year of the mooc. The New York Times, 2(12):2012, 2012.Google Scholar
- J.C. Richardson and K. Swan. Examing social presence in online courses in relation to students' perceived learning and satisfaction. 2003.Google Scholar
- D. Sluijsmans, F. Dochy, and G. Moerkerke. Creating a learning environment by using self-, peer- and co-assessment. Learning Environments Research, 1(3):293--319, 1998.Google ScholarCross Ref
- Hugh Somervell. Issues in assessment, enterprise and higher education: the case for self-peer and collaborative assessment. Assessment and Evaluation in Higher Education, 18(3):221--233, 1993.Google ScholarCross Ref
- K. Swan and L.F. Shih. On the nature and development of social presence in online course discussions. Journal of Asynchronous Learning Networks, 9(3):115--136, 2005.Google Scholar
- Terry Tang, Scott Rixner, and Joe Warren. An environment for learning interactive programming. In Proceedings of the 45th ACM technical symposium on Computer science education, pages 671--676. ACM, 2014. Google ScholarDigital Library
- K. Topping. Peer assessment between students in colleges and universities. Review of Educational Research, 68(3):249--276, 1998.Google ScholarCross Ref
Index Terms
- Grading the Graders: Motivating Peer Graders in a MOOC
Comments