research-article

Developing Feedback Analytics: Discovering Feedback Patterns in an Introductory Course

Author:

Richard GlasseyAuthors Info & Claims

CompEd '19: Proceedings of the ACM Conference on Global Computing Education

Pages 37 - 43

https://doi.org/10.1145/3300115.3309519

Published: 09 May 2019 Publication History

Abstract

Feedback plays a vital role in learning. The earlier misconceptions are identified and corrected, the better for the development of a student's knowledge. Whilst this primarily benefits the student, this paper investigates the value of developing feedback analytics in order to discover new insights, patterns or issues within a course. To facilitate this, issue tracking was used within a large introductory computer science course to capture weekly feedback between teaching assistants and students. Key areas of feedback analytics were identified in terms of the amount of interaction and volume of discussion generated for weekly assignments over a twelve week period. These areas were further divided by focusing on the difference between weekly assignments and the teaching assistants. Applied to two years of data, feedback analytics revealed novel and useful insights about the course, its assignments and the interaction between students and teaching assistants, which helped to highlight areas for deeper course analysis and improvement.

References

[1]

Richard Bailey and Mark Garner. 2010. Is the feedback in higher education assessment worth the paper it is written on? Teachers' reflections on their practices. Teaching in Higher Education, Vol. 15, 2 (2010), 187--198.

[2]

Hannah Blau and J Eliot B Moss. 2015. FrenchPress Gives Students Automated Feedback on Java Program Flaws. ITiCSE '15 (2015), 15--20.

Digital Library

[3]

Tony Dowden, Sharon Pittaway, Helen Yost, and Robyn McCarthy. 2013. Students' perceptions of written feedback in teacher education: Ideally feedback is a continuing two-way communication that encourages progress. Assessment and Evaluation in Higher Education, Vol. 38, 3 (2013), 349--362.

[4]

Joseph Feliciano, Margaret-Anne Storey, and Alexey Zagalsky. 2016. Student experiences using GitHub in software engineering courses: a case study. Proceedings of the 38th International Conference on Software Engineering Companion - ICSE '16 (2016), 422--431.

Digital Library

[5]

Jeffrey Forbes, David J. Malan, Heather Pon-Barry, Stuart Reges, and Mehran Sahami. 2017. Scaling Introductory Courses Using Undergraduate Teaching Assistants. Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education - SIGCSE '17 (2017), 657--658.

Digital Library

[6]

Lassi Haaranen and Teemu Lehtinen. 2015. Teaching Git on the Side: Version Control System As a Course Platform. In Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education. 87--92.

Digital Library

[7]

Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2016. Towards a Systematic Review of Automated Feedback Generation for Programming Exercises. Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education - ITiCSE '16 March (2016), 41--46.

Digital Library

[8]

Chang Liu. 2005. Using issue tracking tools to facilitate student learning of communication skills in software engineering courses. In Proceedings - 18th Conference on Software Engineering Education and Training, CSEE and T 2005. 61--68.

Digital Library

[9]

Ivan Milentijevic, Vladimir Ciric, and Oliver Vojinovic. 2008. Version control in project-based learning. Computers and Education, Vol. 50, 4 (2008), 1331--1338.

Digital Library

[10]

Valbona Muzaka. 2009. The niche of graduate teaching assistants (GTAs): Perceptions and reflections. Teaching in Higher Education, Vol. 14, 1 (2009), 1--12.

[11]

David Nicol. 2010. From monologue to dialogue: Improving written feedback processes in mass higher education. Assessment and Evaluation in Higher Education, Vol. 35, 5 (2010), 501--517.

[12]

David J. Nicol and Debra Macfarlane-Dick. 2006. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, Vol. 31, 2 (2006), 199--218.

[13]

Chris Park and Marife Ramos. 2002. The donkey in the department? Insights into the graduate teaching assistant (GTA) experience in the UK. Journal of Graduate Education, Vol. 3, 2001 (2002), 47--53. http://eprints.lancs.ac.uk/121/

[14]

Margaret Price, Karen Handley, Jill Millar, and Berry O'Donovan. 2010. Feedback: All that effort, but what is the effect? Assessment and Evaluation in Higher Education, Vol. 35, 3 (2010), 277--289.

[15]

Stuart Reges. 2003. Using undergraduates as teaching assistants at a state university. ACM SIGCSE Bulletin, Vol. 35, 1 (2003), 103.

Digital Library

[16]

Kelly Rivers and Kenneth R. Koedinger. 2013. Automatic generation of programming feedback: A data-driven approach. In CEUR Workshop, Vol. 1009.

[17]

Eric Roberts, John Lilly, and Bryan Rollins. 1995. Using undergraduates as teaching assistants in introductory programming courses. ACM SIGCSE Bulletin, Vol. 27, 1 (1995), 48--52.

Digital Library

[18]

D. R. Sadler. 1989. Formative assessment and the design of instructional systems. Instructional science, Vol. 18, 2 (1989), 119---144.

[19]

Michael James Scott and Gheorghita Ghinea. 2015. Reliability in the Assessment of Program Quality by Teaching Assistants During Code Reviews. Proceedings of the 2015 ACM Conference on Innovation and Technology in Computer Science Education - ITiCSE '15, Vol. 85, 6 (2015), 346--346.

Digital Library

[20]

Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. 2013. Automated feedback generation for introductory programming assignments. Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation - PLDI '13 (2013).

Digital Library

[21]

P. Young, V. Yip, and R. B. Lenin. 2012. Evaluation of Issue-tracker's Effectiveness for Measuring Individual Performance on Group Projects. In Proceedings of the 50th Annual Southeast Regional Conference (ACM-SE '12). ACM, New York, NY, USA, 89--94.

Digital Library

Cited By

Berg AOsnes SGlassey RMerkle LDoyle MSheard JSoh LDorn B(2022)If in Doubt, Try ThreeProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499371(362-368)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499371
Conceicao WHolanda MMacedo FIshikawa ENunes VDa Silva D(2022)Automatic Feedback in the Teaching of Programming in Undergraduate Courses: a Literature Mapping2022 IEEE Frontiers in Education Conference (FIE)10.1109/FIE56618.2022.9962723(1-9)Online publication date: 8-Oct-2022
https://doi.org/10.1109/FIE56618.2022.9962723
Sabin MBuffardi KLarsen SMacKellar B(2020)Panel: Synergistic Perspectives on Collaborative Learning with Version Control Tools2020 IEEE Frontiers in Education Conference (FIE)10.1109/FIE44824.2020.9273944(1-4)Online publication date: 21-Oct-2020
https://doi.org/10.1109/FIE44824.2020.9273944

Index Terms

Developing Feedback Analytics: Discovering Feedback Patterns in an Introductory Course
1. Social and professional topics
  1. Professional topics
    1. Computing education

Recommendations

A case study of the development of CS teaching assistants and their experiences with team teaching
Koli Calling '13: Proceedings of the 13th Koli Calling International Conference on Computing Education Research

Teaching assistants play a vital role in lab-based teaching at large institutions, with a large impact on students' success in CS1. How do TAs develop as teachers? We extended existing models of teacher development for our context of teaching CS labs in ...
A Practical Strategy for Training Graduate CS Teaching Assistants to Provide Effective Feedback
ITiCSE 2023: Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1

Computer science (CS) relies heavily on teaching assistants (TAs) who are often untrained in CS pedagogy. Existing research on CS TA training typically studies American undergraduate TAs at high-resource universities, ignoring the many universities that ...
Can We Build an Excellent Undergraduate TA Program? Fostering Collaboration and Developing Best Practices
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2

As the number of students taking Computer Science courses continues to rise, the need for excellent teaching assistants (TAs), becomes increasingly important. In response, numerous CS departments have turned to their expanding pool of undergraduate ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CompEd '19: Proceedings of the ACM Conference on Global Computing Education

May 2019

260 pages

ISBN:9781450362597

DOI:10.1145/3300115

General Chairs:
Ming Zhang
Peking University, China
,
Bo Yang
Linyi University, China
,
Program Chairs:
Steve Cooper
University of Nebraska-Lincoln, USA
,
Andrew Luxton-Reilly
University of Auckland, New Zealand

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CompEd '19

Sponsor:

SIGCSE

CompEd '19: ACM Global Computing Education Conference 2019

May 9 - 19, 2019

Chengdu,Sichuan, China

Acceptance Rates

CompEd '19 Paper Acceptance Rate 33 of 100 submissions, 33%;

Overall Acceptance Rate 33 of 100 submissions, 33%

Upcoming Conference

CompEd '25

Sponsor:
sigcse

ACM Global Computing Education Conference 2025

October 21 - 25, 2025

Gaborone , Botswana

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
185
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Berg AOsnes SGlassey RMerkle LDoyle MSheard JSoh LDorn B(2022)If in Doubt, Try ThreeProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499371(362-368)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499371
Conceicao WHolanda MMacedo FIshikawa ENunes VDa Silva D(2022)Automatic Feedback in the Teaching of Programming in Undergraduate Courses: a Literature Mapping2022 IEEE Frontiers in Education Conference (FIE)10.1109/FIE56618.2022.9962723(1-9)Online publication date: 8-Oct-2022
https://doi.org/10.1109/FIE56618.2022.9962723
Sabin MBuffardi KLarsen SMacKellar B(2020)Panel: Synergistic Perspectives on Collaborative Learning with Version Control Tools2020 IEEE Frontiers in Education Conference (FIE)10.1109/FIE44824.2020.9273944(1-4)Online publication date: 21-Oct-2020
https://doi.org/10.1109/FIE44824.2020.9273944

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten