skip to main content
10.1145/2658779.2658809acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
poster

OverCode: visualizing variation in student solutions to programming problems at scale

Published: 05 October 2014 Publication History

Abstract

In MOOCs, a single programming exercise may produce thousands of solutions from learners. Understanding solution variation is important for providing appropriate feedback to students at scale. The wide variation among these solutions can be a source of pedagogically valuable examples, and can be used to refine the autograder for the exercise by exposing corner cases. We present OverCode, a system for visualizing and exploring thousands of programming solutions. OverCode uses both static and dynamic analysis to cluster similar solutions, and lets instructors further filter and cluster solutions based on different criteria. We evaluated OverCode against a non-clustering baseline in a within-subjects study with 24 teaching assistants, and found that the OverCode interface allows teachers to more quickly develop a high-level view of students' understanding and misconceptions, and to provide feedback that is relevant to more students.

References

[1]
Basu, S., Jacobs, C., and Vanderwende, L. Powergrading: a clustering approach to amplify human effort for short answer grading. TACL 1 (2013), 391--402.
[2]
Gaudencio, M., Dantas, A., and Guerrero, D. D. Can computers compare student code solutions as well as teachers? In Proceedings of the 45th ACM Technical Symposium on Computer Science Education, SIGCSE '14, ACM (New York, NY, USA, 2014), 21--26.
[3]
Huang, J., Piech, C., Nguyen, A., and Guibas, L. J. Syntactic and functional variability of a million code submissions in a machine learning mooc. In AIED Workshops (2013).
[4]
Marton, F., Tsui, A., Chik, P., Ko, P., and Lo, M. Classroom Discourse and the Space of Learning. Taylor & Francis, 2013.
[5]
Nguyen, A., Piech, C., Huang, J., and Guibas, L. J. Codewebs: scalable homework search for massive open online programming courses. In WWW (2014), 491--502.

Cited By

View all
  • (2023)RunEx: Augmenting Regular-Expression Code Search with Runtime Values2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00024(139-147)Online publication date: 3-Oct-2023
  • (2023)Agile Teaching: Automated Student Support and Feedback Generation2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343212(1-9)Online publication date: 18-Oct-2023
  • (2018)Automated clustering and program repair for introductory programming assignmentsACM SIGPLAN Notices10.1145/3296979.319238753:4(465-480)Online publication date: 11-Jun-2018
  • Show More Cited By

Index Terms

  1. OverCode: visualizing variation in student solutions to programming problems at scale

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '14 Adjunct: Adjunct Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology
    October 2014
    150 pages
    ISBN:9781450330688
    DOI:10.1145/2658779
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 October 2014

    Check for updates

    Author Tags

    1. MOOC
    2. data mining
    3. programming exercises

    Qualifiers

    • Poster

    Funding Sources

    Conference

    UIST '14

    Acceptance Rates

    UIST '14 Adjunct Paper Acceptance Rate 74 of 333 submissions, 22%;
    Overall Acceptance Rate 355 of 1,733 submissions, 20%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)6
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 23 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)RunEx: Augmenting Regular-Expression Code Search with Runtime Values2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00024(139-147)Online publication date: 3-Oct-2023
    • (2023)Agile Teaching: Automated Student Support and Feedback Generation2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343212(1-9)Online publication date: 18-Oct-2023
    • (2018)Automated clustering and program repair for introductory programming assignmentsACM SIGPLAN Notices10.1145/3296979.319238753:4(465-480)Online publication date: 11-Jun-2018
    • (2018)Automated clustering and program repair for introductory programming assignmentsProceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation10.1145/3192366.3192387(465-480)Online publication date: 11-Jun-2018
    • (2016)MetaProceedings of the 29th Annual Symposium on User Interface Software and Technology10.1145/2984511.2984532(259-270)Online publication date: 16-Oct-2016
    • (2016)SolMinerProceedings of the 38th International Conference on Software Engineering Companion10.1145/2889160.2889202(481-490)Online publication date: 14-May-2016

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media