skip to main content
10.1145/3341525.3387362acmconferencesArticle/Chapter ViewAbstractPublication PagesiticseConference Proceedingsconference-collections
research-article

CompareCFG: Providing Visual Feedback on Code Quality Using Control Flow Graphs

Published: 15 June 2020 Publication History

Abstract

The quality of the code impacts the cost of its maintenance, yet "code quality" is often not given attention in introductory programming courses, perhaps due to the difficulty of providing automated code quality feedback. We have been exploring how to provide automated feedback on complexity, one aspect of code quality. We have developed CompareCFG that provides feedback based on control flow graphs (CFGs). It generates visualisations of students' submissions and provides the means for a student to compare the CFG of their own code with CFGs of less complex submissions, helping to support their understanding of code complexity. CompareCFG also provides actionable feedback by indicating specific issues in a submission that can reduce its complexity. We evaluated CompareCFG in a pilot study. We found it provides useful feedback to participants that helped them reduce the complexity of their code. CompareCFG offers a convenient way to provide programming students with automated visual feedback on code quality.

References

[1]
Efthimia Aivaloglou and Felienne Hermans. 2016. How Kids Code and How We Know: An Exploratory Study on the Scratch Repository. In Proceedings of the 2016 ACM Conference on International Computing Education Research (ICER '16). Association for Computing Machinery, New York, NY, USA, 53--61. https://doi.org/10.1145/2960310.2960325
[2]
Eliane Araujo, Dalton Serey, and Jorge Figueiredo. 2016. Qualitative aspects of students' programs: Can we make them measurable?. In 2016 IEEE Frontiers in Education Conference (FIE). 1--8. https://doi.org/10.1109/FIE.2016.7757725
[3]
Brett A. Becker and Thomas Fitzpatrick. 2019. What Do CS1 Syllabi Reveal About Our Expectations of Introductory Programming Students?. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE '19). ACM, New York, NY, USA, 1011--1017. https://doi.org/10.1145/3287324.3287485
[4]
Dennis M. Breuker, Jan Derriks, and Jacob Brunekreef. 2011. Measuring Static Quality of Student Code. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education (ITiCSE '11). Association for Computing Machinery, New York, NY, USA, 13--17. https://doi.org/10.1145/1999747.1999754
[5]
Giuseppe De Ruvo, Ewan Tempero, Andrew Luxton-Reilly, Gerard B. Rowe, and Nasser Giacaman. 2018. Understanding Semantic Style by Analysing Student Code. In Proceedings of the 20th Australasian Computing Education Conference (ACE '18). ACM, New York, NY, USA, 73--82. https://doi.org/10.1145/3160489.3160500
[6]
Tomche Delev and Dejan Gjorgjevikj. 2017. Static analysis of source code written by novice programmers. In 2017 IEEE Global Engineering Education Conference (EDUCON). 825--830. https://doi.org/10.1109/EDUCON.2017.7942942
[7]
Paul Denny, Brett A. Becker, Michelle Craig, Greg Wilson, and Piotr Banaszkiewicz. 2019. Research This! Questions That Computing Educators Most Want Computing Education Researchers to Answer. In Proceedings of the 2019 ACM Conference on International Computing Education Research (ICER '19). Association for Computing Machinery, New York, NY, USA, 259--267. https://doi.org/10.1145/3291279.3339402
[8]
Stephen Edwards, Jaime Spacco, and David Hovemeyer. 2019. Can Industrial-Strength Static Analysis Be Used to Help Students Who Are Struggling to Complete Programming Activities?. In Proceedings of the 52nd Hawaii International Conference on System Sciences. 10. https://doi.org/10.24251/HICSS.2019.941
[9]
Stephen H. Edwards, Nischel Kandru, and Mukund B.M. Rajagopal. 2017. Investigating Static Analysis Errors in Student Java Programs. In Proceedings of the 2017 ACM Conference on International Computing Education Research (ICER '17). Association for Computing Machinery, New York, NY, USA, 65--73. https://doi.org/10.1145/3105726.3106182
[10]
Martin Fowler. 2017. Code Smell. http://martinfowler.com/bliki/CodeSmell.html. [Online; accessed September 2017].
[11]
Martin Fowler and Kent Beck. 1999. Refactoring: improving the design of existing code. Addison-Wesley Professional (1999).
[12]
David Hovemeyer, Arto Hellas, Andrew Petersen, and Jaime Spacco. 2016. Control-Flow-Only Abstract Syntax Trees for Analyzing Students' Programming Progress. In Proceedings of the 2016 ACM Conference on International Computing Education Research (ICER '16). Association for Computing Machinery, New York, NY, USA, 63--72. https://doi.org/10.1145/2960310.2960326
[13]
Hieke Keuning, Bastiaan Heeren, and Johan Jeuring. 2017. Code Quality Issues in Student Programs. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '17). ACM, New York, NY, USA, 110--115. https://doi.org/10.1145/3059009.3059061
[14]
Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2016. Towards a Systematic Review of Automated Feedback Generation for Programming Exercises. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE '16). ACM, New York, NY, USA, 41--46. https://doi.org/10.1145/2899415.2899422
[15]
Diana Kirk, Tyne Crow, Andrew Luxton-Reilly, and Ewan Tempero. 2020. On Assuring Learning About Code Quality. In Proceedings of the Twenty-Second Australasian Computing Education Conference (ACE'20). Association for Computing Machinery, New York, NY, USA, 86--94. https://doi.org/10.1145/3373165.3373175
[16]
Andrew Luxton-Reilly, Paul Denny, Diana Kirk, Ewan Tempero, and Se Young Yu. 2013. On the Differences Between Correct Student Solutions. In ACM SIGCSE Conference on Innovation and Technology in Computer Science Education (ITiCSE). Canterbury, United Kingdon, 177--182. http://dx.doi.org/10.1145/2462476.2462505
[17]
Thomas J McCabe. 1976. A complexity measure. IEEE Transactions on software Engineering, Vol. 2, 4 (1976), 308--320.
[18]
Susan A. Mengel and Vinay Yerramilli. 1999. A Case Study of the Static Analysis of the Quality of Novice Student Programs. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99). Association for Computing Machinery, New York, NY, USA, 78--82. https://doi.org/10.1145/299649.299689
[19]
Raymond Pettit, John Homer, Roger Gee, Susan Mengel, and Adam Starbuck. 2015. An Empirical Study of Iterative Improvement in Programming Assignments. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (SIGCSE '15). Association for Computing Machinery, New York, NY, USA, 410--415. https://doi.org/10.1145/2676723.2677279
[20]
Rohan Roy Choudhury, Hezheng Yin, and Armando Fox. 2016. Scale-Driven Automatic Hint Generation for Coding Style. In Intelligent Tutoring Systems, Alessandro Micarelli, John Stamper, and Kitty Panourgia (Eds.). Springer International Publishing, Cham, 122--132.
[21]
Stephen R. Schach. 1999. Classical and Object-Oriented Software Engineering fourth ed.). McGraw-Hill.
[22]
M. Shepperd. 1988. A critique of cyclomatic complexity as a software metric. Software Engineering Journal, Vol. 3, 2 (March 1988), 30--36. https://doi.org/10.1049/sej.1988.0003
[23]
Ian Sommerville. 2011. Software Engineering 9 ed.). Pearson.
[24]
Peeratham Techapalokul and Eli Tilevich. 2017. Novice Programmers and Software Quality: Trends and Implications. In 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE T). 246--250. https://doi.org/10.1109/CSEET.2017.47

Cited By

View all
  • (2025)Introducing Code Quality at CS1 Level: Examples and Activities2024 Working Group Reports on Innovation and Technology in Computer Science Education10.1145/3689187.3709615(339-377)Online publication date: 22-Jan-2025
  • (2025)Teaching Program Decomposition in CS1: A Conceptual Framework for Improved Code QualityProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701880(443-449)Online publication date: 12-Feb-2025
  • (2024)CMCS: contrastive-metric learning via vector-level sampling and augmentation for code searchScientific Reports10.1038/s41598-024-64205-214:1Online publication date: 24-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ITiCSE '20: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education
June 2020
615 pages
ISBN:9781450368742
DOI:10.1145/3341525
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. CFG
  2. automatic assessment tools
  3. code quality
  4. control flow graph
  5. software maintenance

Qualifiers

  • Research-article

Conference

ITiCSE '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Upcoming Conference

ITiCSE '25
Innovation and Technology in Computer Science Education
June 27 - July 2, 2025
Nijmegen , Netherlands

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)45
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Introducing Code Quality at CS1 Level: Examples and Activities2024 Working Group Reports on Innovation and Technology in Computer Science Education10.1145/3689187.3709615(339-377)Online publication date: 22-Jan-2025
  • (2025)Teaching Program Decomposition in CS1: A Conceptual Framework for Improved Code QualityProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701880(443-449)Online publication date: 12-Feb-2025
  • (2024)CMCS: contrastive-metric learning via vector-level sampling and augmentation for code searchScientific Reports10.1038/s41598-024-64205-214:1Online publication date: 24-Jun-2024
  • (2023)Helping to provide adaptive feedback to novice programmers: a framework to assist the Teachers2023 18th Iberian Conference on Information Systems and Technologies (CISTI)10.23919/CISTI58278.2023.10212000(1-6)Online publication date: 20-Jun-2023
  • (2023)Exploring CS1 Student's Notions of Code QualityProceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 110.1145/3587102.3588808(12-18)Online publication date: 29-Jun-2023
  • (2023)FlowARP - Using Augmented Reality for Visualizing Control Flows in ProgramsProceedings of the ACM Conference on Global Computing Education Vol 110.1145/3576882.3617922(161-167)Online publication date: 5-Dec-2023
  • (2023)A Taxonomy to Assist TAs in Providing Adaptive Feedback to Novice Programmers2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343309(1-9)Online publication date: 18-Oct-2023
  • (2022)HSAS-MD Analyzer: A Hybrid Security Analysis System Using Model-Checking Technique and Deep Learning for Malware Detection in IoT AppsSensors10.3390/s2203107922:3(1079)Online publication date: 29-Jan-2022
  • (2022)Automatically Generated Visual Profiles of Code Solutions as Feedback for StudentsInformation10.3390/info1309041513:9(415)Online publication date: 1-Sep-2022
  • (2022)XCode: Towards Cross-Language Code Representation with Large-Scale Pre-TrainingACM Transactions on Software Engineering and Methodology10.1145/350669631:3(1-44)Online publication date: 9-Apr-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media