ABSTRACT
Atoms of confusion are small patterns of code that have been empirically validated to be difficult to hand-evaluate by programmers. Previous research focused on defining and quantifying this phenomenon, but not on explaining or critiquing it. In this work, we address core omissions to the body of work on atoms of confusion, focusing on the ‘how’ and ‘why’ of programmer misunderstanding.
We performed a think-aloud study in which we observed programmers, both professionals and students, as they hand-evaluated confusing code. We performed a qualitative analysis of the data and found several surprising results, which explain previous results, outline avenues of further research, and suggest improvements of the research methodology.
A notable observation is that correct hand-evaluations do not imply understanding, and incorrect evaluations not misunderstanding. We believe this and other observations may be used to improve future studies and models of program comprehension. We argue that thinking of confusion as an atomic construct may pose challenges to formulating new candidates for atoms of confusion. Ultimately, we question whether hand-evaluation correctness is, itself, a sufficient instrument to study program comprehension.
Supplemental Material
- Marwen Abbes, Foutse Khomh, Yann-Gael Gueheneuc, and Giuliano Antoniol. 2011. An empirical study of the impact of two antipatterns, Blob and Spaghetti code, on program comprehension. In Proceedings of the European Conference on Software Maintenance and Reengineering. IEEE, 181ś190.Google ScholarDigital Library
- Mahnaz Behroozi, Alison Lui, Ian Moore, Denae Ford, and Chris Parnin. 2018. Dazed: Measuring the cognitive load of solving technical interview problems at the whiteboard. In Proceedings of the International Conference on Software Engineering : New Ideas and Emerging Technologies Results (ICSE-NIER). IEEE, 93ś96.Google ScholarDigital Library
- Teresa Busjahn, Carsten Schulte, and Andreas Busjahn. 2011. Analysis of code reading to gain more insight in program comprehension. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research. ACM, 1ś9.Google ScholarDigital Library
- Fernando Castor. 2018. Identifying confusing code in Swift programs. In Proceedings of the VI CBSoft Workshop on Visualization, Evolution, and Maintenance. ACM.Google Scholar
- Elizabeth Charters. 2003. The use of think-aloud methods in qualitative research an introduction to think-aloud methods. Brock Education: A Journal of Educational Research and Practice 12, 2 ( 2003 ), 68ś82.Google Scholar
- Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Sage Publications.Google Scholar
- John W. Creswell and J. David Creswell. 2017. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications.Google Scholar
- Beth Davey. 1983. Think aloud: Modeling the cognitive processes of reading comprehension. Journal of Reading 27, 1 ( 1983 ), 44ś47.Google Scholar
- Barney G. Glaser and Anselm L. Strauss. 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine de Gruyter.Google Scholar
- GCC GNU Compiler Collection. (accessed March 4, 2020 ). The C Preprocessor: Undefining and Redefining Macros. https://gcc.gnu.org/onlinedocs/cpp/Undefiningand-Redefining-Macros.htmlGoogle Scholar
- Dan Gopstein, Jake Iannacone, Yu Yan, Lois Anne Delong, Yanyan Zhuang, Martin K.-C. Yeh, and Justin Cappos. 2017. Understanding misunderstandings in source code. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. ACM, 129ś139.Google ScholarDigital Library
- Dan Gopstein, Hongwei Henry Zhou, Phyllis Frankl, and Justin Cappos. 2018. Prevalence of confusing code in software projects: Atoms of Confusion in the wild. In Proceedings of the International Conference on Mining Software Repositories. ACM, 11 pages.Google ScholarDigital Library
- ISO. 1999. ISO/IEC 9899: 1999 : Programming Languages Ð C. 538 pages.Google Scholar
- ISO. 2011. ISO/IEC 14882: 2011 Information technology Ð Programming languages Ð C++ (third ed.). http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail. htm?csnumber=50372Google Scholar
- Brian W Kernighan and Dennis M Ritchie. 1998. The C Programming LanguageANSI C Version.Google Scholar
- Jörg Liebig, Christian Kästner, and Sven Apel. 2011. Analyzing the discipline of preprocessor annotations in 30 million lines of C code. In Proceedings of the International Conference on Aspect-Oriented Software Development. ACM, 191ś 202.Google ScholarDigital Library
- Flávio Medeiros, Gabriel Lima, Guilherme Amaral, Sven Apel, Christian Kästner, Márcio Ribeiro, and Rohit Gheyi. 2019. An investigation of misunderstanding code patterns in C open-source software projects. Empirical Software Engineering 24, 4 ( 2019 ), 1693ś1726.Google Scholar
- Tobias Röhm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej. 2012. How do professional developers comprehend software?. In Proceedings of the International Conference on Software Engineering. IEEE, 255ś265.Google ScholarCross Ref
- Carolyn B. Seaman. 1999. Qualitative methods in empirical studies of software engineering. IEEE Transactions on Software Engineering 25, 4 ( 1999 ), 557ś572.Google ScholarDigital Library
- Janet Siegmund, Norbert Siegmund, and Sven Apel. 2015. Views on internal and external validity in empirical software engineering. In Proceedings of the International Conference on Software Engineering, Vol. 1. IEEE, 9ś19.Google ScholarCross Ref
- Maarten W. Van Someren, Yvonne F. Barnard, and Jacobijn A. C. Sandberg. 1994. The Think Aloud Method: A Practical Approach to Modelling Cognitive Processes. Academic Press Inc.Google Scholar
- Klaas-Jan Stol, Paul Ralph, and Brian Fitzgerald. 2016. Grounded theory in software engineering research: A critical review and guidelines. In Proceedings of the International Conference on Software Engineering. IEEE, 120ś131.Google ScholarDigital Library
- Anselm L. Strauss. 1987. Qualitative Analysis for Social Scientists. Cambridge University Press.Google Scholar
- Susan Wiedenbeck. 1986. Beacons in computer program comprehension. International Journal of Man-Machine Studies 25, 6 ( 1986 ), 697ś709.Google ScholarDigital Library
- Aiko Yamashita and Leon Moonen. 2013. Towards a taxonomy of programmingrelated dificulties during maintenance. In Proceedings of the International Conference on Software Maintenance. IEEE, 424ś427.Google Scholar
Index Terms
- Thinking aloud about confusing code: a qualitative investigation of program comprehension and atoms of confusion
Recommendations
Atoms of Confusion: The Eyes Do Not Lie
SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software EngineeringCode comprehension is crucial in software maintenance activities, though it can be hindered by misunderstandings and confusion patterns, namely, atoms of confusion. They are small pieces of code using specific programming language constructs, such as ...
Understanding misunderstandings in source code
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software EngineeringHumans often mistake the meaning of source code, and so misjudge a program's true behavior. These mistakes can be caused by extremely small, isolated patterns in code, which can lead to significant runtime errors. These patterns are used in large, ...
Prevalence of confusing code in software projects: atoms of confusion in the wild
MSR '18: Proceedings of the 15th International Conference on Mining Software RepositoriesPrior work has shown that extremely small code patterns, such as the conditional operator and implicit type conversion, can cause considerable misunderstanding in programmers. Until now, the real world impact of these patterns - known as 'atoms of ...
Comments