Philip Arcuria
ABSTRACT
Rubrics are widely used throughout postsecondary education as means for aiding in instruction and evaluation. However, despite their broad global adoption, very little is known about the quality of rubrics in use. We develop two measures to assess the quality of rubrics: (1) a checklist identifying criteria of high-quality rubrics based on analytic rubric design best practices and (2) a set of LMS-derived features that are hypothesized to represent structural components that are, in general, necessary but not sufficient for high quality rubrics. The validity of using the feature-generated scores as proxies for identifying rubric quality is evaluated through several means. First, the feature-generated scores are calculated for a set of external exemplary rubrics of known high quality. Second, the feature-scores for a subset of internal rubrics are compared to average human rater scores of rubric quality based on the checklist. We discuss the results, practical applications, and a larger research program surrounding the feature-generated scores.
- Carnegie Mellon University. 2016. Creating and Using Rubrics. Retrieved from www.cmu.edu/teaching/assessment/assesslearning/rubrics.html/.Google Scholar
- Chinese University of Hong Kong. Policy on Assessment of Student Learning in Taught Programmes. Retrieved from www.cuhk.edu.hk/clear/qm/A5-1.pdf/.Google Scholar
- Flinders University. 2009. Using Rubrics in Assessment. Retrieved from www.flinders.edu.au/teaching/teaching-strategies/assessment/grading/using-rubrics.cfm/.Google Scholar
- University of Manchester. 2018. Rubrics -- What Are They? Why and how should I use them? Retrieved from www.elearning.fse.manchester.ac.uk/rubrics-what-are-they-and-why-and-how-should-i-use-them/.Google Scholar
- University of Waterloo. 2018. Rubrics: Useful Assessment Tools. Retrieved from https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/assessing-student-work/grading-and-feedback/rubrics-useful-assessment-tools/.Google Scholar
- Blackboard. 2018. Rubrics. Retrieved from https://help.blackboard.com/Learn/Instructor/Grade/Rubrics/.Google Scholar
- Desire2Learn. 2018. Rubrics. Retrieved from https://documentation.brightspace.com/EN/semester_start/-/instructor/faq_rubrics.htm/.Google Scholar
- Instructure. 2017. How do I Add a Rubric in a Course? Retrieved from https://community.canvaslms.com/docs/DOC-12722-415286227/.Google Scholar
- Moodle. 2018. Rubrics. Retrieved from https://docs.moodle.org/35/en/Rubrics/.Google Scholar
- Susan M. Brookhart. 2013. How to create and use rubrics for formative assessment and grading. Association for Supervision & Curriculum Development, Alexandria, VA.Google Scholar
- W. James Popham. 2000. Modern educational measurement: Practical guidelines for educational leaders (3rd ed.). Allyn and Bacon, Boston, MA.Google Scholar
- Anthony J. Nitko and Susan M. Brookhart. 2007. Educational Assessment of Students (5th ed.). Pearson Education, Upper Saddle River, NJ. Google ScholarDigital Library
- Linda Suskie. 2009. Assessing student learning: A common sense guide (2nd ed). Jossey-Bass, San Francisco, CA.Google Scholar
- Trudy W. Banta and Catherine A. Palomba, C. A. 2015. Assessment essentials: Planning, implementing, and improving assessment in higher education. Jossey-Bass, San Francisco, CA.Google Scholar
- Heidi Goodrich Andrade. 2005. Teaching with rubrics: The good, the bad, and the ugly. College Teaching, 53, 1 (2005), 27-32.Google ScholarCross Ref
- Andrea Cockett and Carole Jackson. 2018. The use of assessment rubrics to enhance feedback in higher education: An integrated literature review. Nurse Education Today, 69 (2018), 8--13.Google ScholarCross Ref
- Sebastian Barney, Mahvish Khurum, Kai Petersen, Michael Unterkalmsteiner, and Ronald Jabangwe. 2012. Improving students with rubric-based self-assessment and oral feedback. IEEE Transactions on Education, 55, 3 (2012), 319--325. Google ScholarDigital Library
- Susan M. Brookhart and Fei Chen. 2015. The quality and effectiveness of descriptive rubrics. Educational Review, 67, 3 (2015), 343--368.Google ScholarCross Ref
- Katherine P. Greenberg. 2015. Rubric use in formative assessment: A detailed behavioral rubric helps students improve their scientific writing skills. Teaching of Psychology, 42, 3 (2015), 211--217.Google ScholarCross Ref
- Doncho Petkov and Olga Petkova. 2006. Development of scoring rubrics for IS projects as an assessment tool. Issues in Informing Science and Information Technology, 3 (2006), 499--510.Google ScholarCross Ref
- Rebecca J. Howell. 2014. Grading rubrics: hoopla or help? Innovations in Education and Teaching International, 51, 4 (2014), 400--410.Google ScholarCross Ref
- Yu Huang and Min Gui. 2015. Articulating teachers' expectations afore: Impact of rubrics on Chinese EFL Learners' Sefl-Assessment and Speaking Ability. Journal of Educational and Training Studies, 3, 3 (2015), 126--132.Google Scholar
- Y. Malini Reddy and Heidi Andrade. 2010. A Review of Rubric Use in Higher Education. Assessment & Evaluation in Higher Education, 35, 4 (2010), 435--448.Google ScholarCross Ref
- C. A. Reitmeier, L. K. Svendsen, and D. A. Vrchota. 2004. Journal of Food Science Education, 3 (2004), 15--20.Google Scholar
- Mark P. J. van der Loo. 2014. The stringdist package for approximate string matching. The R Journal, 6 (2014), 111--122.Google ScholarCross Ref
- Heidi Andrade and Ying Du. 2005. Student perspectives on rubric-referenced assessment. Practical Assessment, Research and Eval., 10, 3 (2005), 1--11.Google Scholar
- AACU: American Association of College and Universities. 2017. VALUE rubric development project. Retrieved from https://www.aacu.org/value/rubrics/.Google Scholar
- Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 10, 1 (1960), 37--46.Google ScholarCross Ref
Index Terms
- Validating the Use of LMS-Derived Rubric Structural Features to Facilitate Automated Measurement of Rubric Quality
Recommendations
Utilization of Rubrics for Self-assessment of Computer Science Students Enrolled in a Research Writing Course
WCCCE '17: Proceedings of the 22nd Western Canadian Conference on Computing EducationThis study 1 determined the utilization of rubrics in self-assessment of Computer Science students who underwent research writing course. Two sections with a total of 69 students served as participants of the study. The grades of the teacher in the ...
Designing a rubric for feedback on code quality in programming courses
Koli Calling '16: Proceedings of the 16th Koli Calling International Conference on Computing Education ResearchWe investigate how to create a rubric that can be used to give feedback on code quality to students in introductory programming courses. Based on an existing model of code quality and a set of preliminary design rules, we constructed a rubric and put it ...
Logarithmic rubric scoring
Papers of the 32nd Annual CCSC South Central ConferenceGrading rubrics communicate convenient explicitly articulated performance expectations to students and provide a consistent methodology for assessment, but using linearly weighted rubrics can lead to poor grade differentiation among high performing ...
Comments