research-article

On the Comprehension of Program Comprehension

Authors:

Rebecca Tiarks,

Rainer KoschkeAuthors Info & Claims

ACM Transactions on Software Engineering and Methodology (TOSEM), Volume 23, Issue 4

Article No.: 31, Pages 1 - 37

https://doi.org/10.1145/2622669

Published: 05 September 2014 Publication History

Abstract

Research in program comprehension has evolved considerably over the past decades. However, only little is known about how developers practice program comprehension in their daily work. This article reports on qualitative and quantitative research to comprehend the strategies, tools, and knowledge used for program comprehension. We observed 28 professional developers, focusing on their comprehension behavior, strategies followed, and tools used. In an online survey with 1,477 respondents, we analyzed the importance of certain types of knowledge for comprehension and where developers typically access and share this knowledge.

We found that developers follow pragmatic comprehension strategies depending on context. They try to avoid comprehension whenever possible and often put themselves in the role of users by inspecting graphical interfaces. Participants confirmed that standards, experience, and personal communication facilitate comprehension. The team size, its distribution, and open-source experience influence their knowledge sharing and access behavior. While face-to-face communication is preferred for accessing knowledge, knowledge is frequently shared in informal comments.

Our results reveal a gap between research and practice, as we did not observe any use of comprehension tools and developers seem to be unaware of them. Overall, our findings call for reconsidering the research agendas towards context-aware tool support.

References

[1]

Mark Ackerman, Volkmar Pipek, and Volker Wulf. 2003. Sharing Expertise Beyond Knowledge Management. The MIT Press.

Digital Library

[2]

B. Adelson and E. Soloway. 1985. The role of domain expenence in software design. IEEE Trans. Softw. Eng. 11, 11 (1985), 1351--1360.

Digital Library

[3]

T. Anderson and H. Kanuka. 2003. e-RESEARCH Methods, Strategies, and Issues. Pearson Education.

[4]

F. O. Bjørnson and T. Dingsøyr. 2008. Knowledge management in software engineering: A systematic review of studied concepts, findings and research methods used. Inf. Softw. Technol. 50, 11 (2008), 1055--1068.

Digital Library

[5]

B. W. Boehm. 1976. Software engineering. IEEE Trans. Comput. C-25, 12 (1976), 1226--1241.

Digital Library

[6]

J. Brandt, M. Dontcheva, M. Weskamp, and S. R. Klemmer. 2010. Example-centric programming: Integrating Web search into the development environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'10). ACM, New York, NY, 513--522.

Digital Library

[7]

J. Brandt, Ph. J. Guo, J. Lewenstein, M. Dontcheva, and S. R. Klemmer. 2009. Two studies of opportunistic programming: Interleaving Web foraging, learning, and writing code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'09). ACM, New York, NY, 1589--1598.

Digital Library

[8]

S. Breu, R. Premraj, J. Sillito, and Th. Zimmermann. 2010. Information needs in bug reports: Improving cooperation between developers and users. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'10). ACM, New York, NY, 301--310.

Digital Library

[9]

R. E. Brooks. 1983. Towards a theory of the comprehension of computer programs. Int. J. Man-Machine Stud. 18, 6 (1983), 543--554.

[10]

A. Cabrera and E. F. Cabrera. 2002. Knowledge-sharing dilemmas. Org. Stud. 23 (2002), 687--710.

[11]

J. Carroll. 1998. Minimalism beyond the Nurnberg Funnel. MIT Press.

[12]

M. Cataldo and J. D. Herbsleb. 2008. Communication networks in geographically distributed software development. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'08). ACM, New York, NY, 579--588.

Digital Library

[13]

J. W. Creswell. 2009. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications, Inc.

[14]

B. Dagenais and M. P. Robillard. 2010. Creating and evolving developer documentation: Understanding the decisions of open source contributors. In Proceedings of the 18th ACM SIGSOFT International Symposium on the Foundations of Software Engineering. 127--136.

Digital Library

[15]

R. DeLine, A. Khella, M. Czerwinski, and G. Robertson. 2005. Towards understanding programs through wear-based filtering. In Proceedings of the ACM Symposium on Software Visualization (SoftVis'05). ACM, 183--192.

Digital Library

[16]

Kevin C. Desouza. 2003. Barriers to effective use of knowledge management systems in software engineering. Commun. ACM 46, 1 (2003), 99--101.

Digital Library

[17]

Francoise Detienne. 2002. Software Design -- Cognitive Aspect. Springer.

Digital Library

[18]

K. A. Ericsson and H. A. Simon. 1993. Protocol Analysis: Verbal Reports as Data. Massachusetts Institute of Technology.

[19]

R. K. Fjeldstad and W. T. Hamlen. 1979. Application program maintenance study: Report to our respondents. In Proceedings of GUIDE 48. PA.

[20]

A. Forward and T. C. Lethbridge. 2002. The relevance of software documentation, tools and technologies: A survey. In Proceedings of the ACM Symposium on Document Engineering. ACM, 26--33.

Digital Library

[21]

Th. Fritz and G. C. Murphy. 2010. Using information fragments to answer the questions developers ask. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (ICSE'10). ACM, New York, NY, 175--184.

Digital Library

[22]

G. Gilder. 2013. Knowledge and Power: The Information Theory of Capitalism and How it is Revolutionizing our World. Regnery Publishing.

[23]

A. M. Grubb and A. Begel. 2012. On the perceived interdependence and information sharing inhibitions of enterprise software engineers. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'12). ACM, New York, NY, 1337--1346.

Digital Library

[24]

J. D. Herbsleb and A. Mockus. 2003. An empirical study of speed and communication in globally-distributed software development. IEEE Trans. Softw. Eng. 29, 6 (2003), 481--494.

Digital Library

[25]

R. Hoffmann, J. Fogarty, and D. S. Weld. 2007. Assieme: Finding and leveraging implicit references in a Web search interface for programmers. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST'07). ACM, New York, NY, 13--22.

Digital Library

[26]

R. Holmes and R. J. Walker. 2013. Systematizing pragmatic software reuse. ACM Trans. Softw. Eng. Methodol. 21, 4, Article 20 (2013).

Digital Library

[27]

Miryung Kim, L. Bergman, T. Lau, and D. Notkin. 2004. An ethnographic study of copy and paste programming practices in OOPL. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE'04). 83--92.

Digital Library

[28]

A. J. Ko, R. DeLine, and G. Venolia. 2007. Information needs in collocated software development teams. In Proceedings of the 29th International Conference on Software Engineering (ICSE'07). IEEE Computer Society, 344--353.

Digital Library

[29]

A. J. Ko and B. A. Myers. 2004. Designing the whyline: A debugging interface for asking questions about program behavior. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'04). ACM, 151--158.

Digital Library

[30]

A. J. Ko and B. A. Myers. 2008. Debugging reinvented: Asking and answering why and why not questions about program behavior. In Proceedings of the 30th International Conference on Software Engineering (ICSE'08). ACM, New York, NY, 301--310.

Digital Library

[31]

A. J. Ko, B. A. Myers, M. J. Coblenz, and H. H. Aung. 2006. An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Trans. Softw. Eng. 32, 12 (2006), 971--987.

Digital Library

[32]

Th. D. LaToza, D. Garlan, J. D. Herbsleb, and B. A. Myers. 2007. Program comprehension as fact finding. In Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC-FSE'07). ACM, New York, NY, 361--370.

Digital Library

[33]

Th. D. LaToza, G. Venolia, and R. DeLine. 2006. Maintaining mental models: A study of developer work habits. In Proceeding of the 28th International Conference on Software Engineering (ICSE'06). ACM, New York, NY, 492--501.

Digital Library

[34]

T. C. Lethbridge, J. Singer, and A. Forward. 2003. How software engineers use documentation: The state of the practice. IEEE Softw. 20, 6 (2003), 35--39.

Digital Library

[35]

S. Letovsky and E. Soloway. 1986. Delocalized plans and program comprehension. IEEE Softw. 3, 3 (1986), 41--49.

Digital Library

[36]

D. C. Littman, J. Pinto, S. Letovsky, and E. Soloway. 1987. Mental models and software maintenance. J. Syst. Softw. 7, 4 (1987), 341--355.

Digital Library

[37]

W. Maalej. 2009. Task-first or context-first&quest; Tool integration revisited. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE'09). IEEE Computer Society, 344--355.

Digital Library

[38]

Walid Maalej, Thomas Fritz, and Romain Robbes. 2014. Collecting and processing interaction data for recommendation systems. In Recommendation Systems in Software Engineering, Martin P. Robillard, Walid Maalej, Robert J. Walker, and Thomas Zimmermann (Eds.), Springer, Berlin, 173--197.

[39]

W. Maalej and H.-J. Happel. 2010. Can development work describe itself&quest;. In Proceedings of the 7th IEEE International Working Conference on Mining Software Repositories (MSR'10). IEEE, 191--200.

[40]

Walid Maalej and Martin P. Robillard. 2013. Patterns of knowledge in API reference documentation. IEEE Trans. Softw. Eng. 39, 9 (2013), 1264--1282.

Digital Library

[41]

A. E. Milewski, M. Tremaine, R. Egan, S. Zhang, F. Kobler, and P. O'Sullivan. 2008. Guidelines for effective bridging in global software engineering. In Proceedings of the IEEE International Conference on Global Software Engineering. IEEE Computer Society, 23--32.

Digital Library

[42]

A. Mockus and J. D. Herbsleb. 2002. Expertise browser: A quantitative approach to identifying expertise. In Proceedings of the 24th International Conference on Software Engineering (ICSE'02). ACM, New York, NY, 503--512.

Digital Library

[43]

G. C. Murphy, M. Kersten, and L. Findlater. 2006. How are Java software developers using the Eclipse IDE&quest; IEEE Softw. 23, 4 (2006), 76--83.

Digital Library

[44]

E. Murphy-Hill. 2014. The future of social learning in software engineering. Computer 47, 1 (2014), 48--54.

Digital Library

[45]

E. Murphy-Hill, R. Jiresal, and G. C. Murphy. 2012a. Improving software developers' fluency by recommending development environment commands. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (FSE'12). ACM, New York, NY, Article 42, 11 pages.

Digital Library

[46]

E. Murphy-Hill and G. C. Murphy. 2011. Peer Interaction effectively yet infrequently, enables programmers to discover new tools. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW'11). ACM, New York, NY, 405--414.

Digital Library

[47]

E. Murphy-Hill, C. Parnin, and A. P. Black. 2012b. How we refactor, and how we know it. IEEE Trans. Softw. Eng. 38, 1 (2012), 5--18.

Digital Library

[48]

Ch. Parnin and S. Rugaber. 2009. Resumption strategies for interrupted programming tasks. In Proceedings of the IEEE 17th International Conference on Program Comprehension (ICPC). IEEE.

[49]

F. Rahman, Ch. Bird, and P. Devanbu. 2012. Clones: What is that smell&quest; Empirical Softw. Eng. 17, 4--5 (2012), 503--530.

Digital Library

[50]

V. Rajlich and N. Wilde. 2002. The role of concepts in program comprehension. In Proceedings of the 10th International Workshop on Program Comprehension. 271--278.

Digital Library

[51]

T. Reichling, M. Veith, and V. Wulf. 2007. Expert recommender: Designing for a Network Organization. Comput. Supported Coop. Work 16, 4--5 (2007), 431--465.

Digital Library

[52]

M. Robillard, W. Coelho, and G. Murphy. 2004. How effective developers investigate source code: An exploratory Study. IEEE Trans. Softw. Eng. 30, 12 (2004), 889--903.

Digital Library

[53]

M. Robillard and R. DeLine. 2010. A field study of API learning obstacles. Empirical Softw. Eng. (2010), 1--30.

Digital Library

[54]

Martin P. Robillard. 2009. What makes APIs hard to learn&quest; Answers from developers. IEEE Softw. 26, 6 (2009), 27--34.

Digital Library

[55]

T. Roehm and W. Maalej. 2012. Automatically detecting developer activities and problems in software development work: NIER track. In Proceedings of the 34th International Conference on Software Engineering, (ICSE'12).

Digital Library

[56]

T. Roehm, R. Tiarks, R. Koschke, and W. Maalej. 2012. How do professional developers comprehend software&quest; In Proceedings of the International Conference on Software Engineering (ICSE'12). IEEE Press, 255--265. http://dl.acm.org/citation.cfm&quest;id=2337223.2337254.

Digital Library

[57]

R. L. Rosnow and R. Rosenthal. 2007. Beginning Behavioral Research: A Conceptual Primer (6th. Ed.). Pearson.

[58]

J. Sillito, K. de Voider, B. Fisher, and G. Murphy. 2005. Managing software change tasks: An exploratory study. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE'05).

[59]

J. Sillito, G. C. Murphy, and K. de Volder. 2008. Asking and answering questions during a programming change task. Trans. Softw. Eng. 34 (2008), 434--451. Issue 4.

Digital Library

[60]

J. Singer, T. Lethbridge, N. Vinson, and N. Anquetil. 1997. An examination of software engineering work practices. In Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research (CASCO). IBM Press, 21.

Digital Library

[61]

L.-G. Singer. 2013. Improving the Adoption of Software Engineering Practices through Persuasive Interventions. lulu.com.

[62]

D. K. Sjøberg, B. Anda, E. Arisholm, T. Dybå, M. Jørgensen, A. Karahasanović, and M. Vokáč. 2003. Challenges and recommendations when increasing the realism of controlled software engineering experiments. In Empirical Methods and Studies in Software Engineering, Reidar Conradi and AlfInge Wang (Eds.), Lecture Notes in Computer Science, vol. 2765, Springer, Berlin, 24--38.

[63]

E. Soloway and K. Ehrlich. 1984. Empirical studies of programming knowledge. IEEE Trans. Softw. Eng. 10, 5 (1984), 595--609.

Digital Library

[64]

Tracy L. Tuten, David J. Urban, and Michael Bosnjak. 2002. Internet Surveys and Data Quality: A Review. Hogrefe and Huber Publishers.

[65]

M. Vakilian, N. Chen, S. Negara, B. A. Rajkumar, B. P. Bailey, and R. E. Johnson. 2012. Use, disuse, and misuse of automated refactorings. In Proceedings of the International Conference on Software Engineering (ICSE'12). IEEE Press, 233--243. http://dl.acm.org/citation.cfm&quest;id=2337223.2337251.

Digital Library

[66]

A. Von Mayrhauser and A. M. Vans. 1996. Identification of dynamic comprehension processes during large scale maintenance. IEEE Trans. Softw. Eng. 22, 6 (1996), 424--437.

Digital Library

[67]

A. Von Mayrhauser, A. M. Vans, and S. Lang. 1998. Program comprehension and enhancement of software. In Proceedings of the IFIP World Computing Congress - Information Technology and Knowledge Engineering.

[68]

J. C. F. D. Winter and D. Dodou. 2010. Five-point Likert items: t test versus Mann-Whitney-Wilcoxon. Practical Assess. Res. Eval. 15, 11 (2010).

Cited By

Ashtari NChilana P(2024)How New Developers Approach Augmented Reality Development Using Simplified Creation Tools: An Observational StudyMultimodal Technologies and Interaction10.3390/mti80400358:4(35)Online publication date: 22-Apr-2024
https://doi.org/10.3390/mti8040035
Yen RZhu JSuh SXia HZhao J(2024)CoLadder: Manipulating Code Generation via Multi-Level BlocksProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676357(1-20)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676357
Yang HChen HKuai ZTu SKuang LBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)ASKDetector: An AST-Semantic and Key Features Fusion based Code Comment Mismatch DetectorProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644405(392-402)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644405
Show More Cited By

Index Terms

On the Comprehension of Program Comprehension
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Software management
        Software maintenance
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues

Recommendations

Analysis of code reading to gain more insight in program comprehension
Koli Calling '11: Proceedings of the 11th Koli Calling International Conference on Computing Education Research

Code reading, although an integral part of program comprehension, is rarely reflected. In this paper, we want to argue for a research approach and direction exploiting the potential that lies in the analysis of reading processes. Based on the vast ...
A comparison of program comprehension strategies by blind and sighted programmers
ICSE '18: Proceedings of the 40th International Conference on Software Engineering

Programmers who are blind use a screen reader to speak source code one word at a time, as though the code were text. For example, "float f = 5.23;" can be read as "float f equals five point two three semicolon". This process of reading is in stark ...
Measuring program comprehension: a large-scale field study with professionals
ICSE '18: Proceedings of the 40th International Conference on Software Engineering

During software development and maintenance, developers spend a considerable amount of time on program comprehension. Previous studies show that program comprehension takes up as much as half of a developer's time. However, most of these studies are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology

ACM Transactions on Software Engineering and Methodology Volume 23, Issue 4

Special Issue International Conference on Software Engineering (ICSE 2012) and Regular Papers

August 2014

232 pages

ISSN:1049-331X

EISSN:1557-7392

DOI:10.1145/2668018

Editors:
David S. Rosenblum
National University of Singapore, Singapore
,
Gail C. Murphy,
Mauro Pezzé

Issue’s Table of Contents

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 September 2014

Accepted: 01 May 2014

Revised: 01 January 2014

Received: 01 December 2012

Published in TOSEM Volume 23, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

124
Total Citations
View Citations
1,862
Total Downloads

Downloads (Last 12 months)248
Downloads (Last 6 weeks)45

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ashtari NChilana P(2024)How New Developers Approach Augmented Reality Development Using Simplified Creation Tools: An Observational StudyMultimodal Technologies and Interaction10.3390/mti80400358:4(35)Online publication date: 22-Apr-2024
https://doi.org/10.3390/mti8040035
Yen RZhu JSuh SXia HZhao J(2024)CoLadder: Manipulating Code Generation via Multi-Level BlocksProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676357(1-20)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676357
Yang HChen HKuai ZTu SKuang LBaysal OLinares-Vasquez MMoran KSteinmacher I(2024)ASKDetector: An AST-Semantic and Key Features Fusion based Code Comment Mismatch DetectorProceedings of the 32nd IEEE/ACM International Conference on Program Comprehension10.1145/3643916.3644405(392-402)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3643916.3644405
Song ZChen JZhang K(2024)Bin2Summary: Beyond Function Name Prediction in Stripped Binaries with Functionality-Specific Code EmbeddingsProceedings of the ACM on Software Engineering10.1145/36437291:FSE(47-69)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643729
Nam DMacvean AMyers BVasilescu B(2024)Understanding Documentation Use Through Log Analysis: A Case Study of Four Cloud ServicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642721(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642721
Horvath AMacvean AMyers B(2024)Meta-Manager: A Tool for Collecting and Exploring Meta Information about CodeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642676(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642676
Nam DMacvean AHellendoorn VVasilescu BMyers BRoychoudhury APaiva AAbreu RStorey M(2024)Using an LLM to Help With Code UnderstandingProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639187(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639187
Krause-Glau AHansen MHasselbring W(2024)Collaborative Design and Planning of Software Architecture Changes via Software City Visualization2024 IEEE Working Conference on Software Visualization (VISSOFT)10.1109/VISSOFT64034.2024.00030(155-156)Online publication date: 6-Oct-2024
https://doi.org/10.1109/VISSOFT64034.2024.00030
Kruse HPuhlfürϐ TMaalej W(2024)Can Developers Prompt? A Controlled Experiment for Code Documentation Generation2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00058(574-586)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00058
Tayeb AAlahmadi MTajik EHaiduc S(2024)Investigating Developers' Preferences for Learning and Issue Resolution Resources in the ChatGPT Era2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00045(413-425)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00045
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents