Skip to main content
SpringerLink
Log in

Discovering undocumented knowledge through visualization of agile software development activities

Case studies on industrial projects using issue tracking system and version control system

  • RE 2017
  • Published:
Requirements Engineering Aims and scope Submit manuscript

Abstract

In agile software development projects, software engineers prioritize implementation over documentation. Is the cost of missing documentation greater than the cost of producing unnecessary or unused documentation? Agile software engineers must still maintain other software artifacts, such as tickets in an issue tracking system or source code committed to a version control system (VCS). Do these artifacts contain useful knowledge? In this paper, we examine undocumented knowledge in a multi-case exploratory case study of industrial agile software development projects. The first is an internal project with 159 source code commits and roughly 8000 lines of code. The second is an external project with 760 source code commits and roughly 50,000 lines of code. We introduce a ticket-commit network chart (TCC) that visually represents time-series commit activities along with filed issue tickets. We also implement a tool to generate the TCC using both commit log and ticket data. Our case study revealed that software engineers committed source code to the VCS without a corresponding issue ticket in a non-trivial minority of instances. If these commits were based on and linked to individual issue tickets, then these “unissued” tickets would have accounted for a non-trivial minority (5–21%) of the knowledge needed for future software modification and operations. End users and requirements engineers also evaluated the contents of these commits. They found that the omission of links to individual tickets had an important impact on future software modification or operation with between 22 and 49% of these instances resulting in undocumented knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. A commit may be unlinked to an issue ticket at the time it is merged into the repository. This could happen for one of two reasons. First, the issue ticket may exist, and the software engineer simply failed to link the commit to it. Second, the issue ticket may not exist. We refer to both cases as “unlinked commits” and to the second cases as “unissued tickets.” Commits should, however, always be linked to an issue ticket because of the explicit methodological desire to connect one commit with one issue ticket in both cases of our study. Even if a commit could have been linked to an already issued ticket, the fact that it was not remains problematic.

  2. Note that we did not interview the software engineers because we felt that they would have been biased in commenting on their own source code commit data.

  3. The original interview was conducted in Japanese. Questions RE-1, RE-2, and EU-1 are English translations.

References

  1. Apache Subversion. Enterprise-class centralized version control for the masses. https://subversion.apache.org/. Accessed 03 Jan 2017

  2. Backlog. Online project management tool for developers. https://backlog.com/. Accessed 03 Jan 2017

  3. Beck K, Cynthia A (2005) Extreme programming explained: embrace change. Addison-Wesley Professional, Boston

    Google Scholar 

  4. Beck K, Grenning J, Martin RC, Beedle M, Highsmith J, Mellor S, van Bennekum A, Hunt A, Schwaber K, Cockburn A, Jefferies R, Sutherland J, Cunningham W, Kern J, Thomas D, Fowler M, Marick B (2001) Manifesto for agile software development. http://agilemanifesto.org/. Accessed 10 Jan 2017

  5. Cockburn A (2006) Agile software development: the cooperative game, 2nd edn. Addison-Wesley Professional, Boston

    Google Scholar 

  6. Cohn M (2005) Agile estimating and planning. Prentice Hall, Upper Saddle River

    Google Scholar 

  7. Dhungana D, Grünbacher P, Rabiser R (2011) The DOPLER meta-tool for decision-oriented variability modeling: a multiple case study. Autom Softw Eng 18(1):77–114

    Article  Google Scholar 

  8. Dorairaj S, Noble J, Malik P (2012) Knowledge management in distributed agile software development. In: The Agile conference, pp 64–73

  9. Eisenhardt KM (1989) Building theories from case study research. Acad Manag Rev 14(4):532–550

    Article  Google Scholar 

  10. GitHub. The world’s leading software development platform. https://github.com/. Accessed 03 Jan 2017

  11. Hoda R, Noble J, Marshall S (2010) How much is just enough?: Some documentation patterns on agile projects. In: Proceedings of the 15th European conference on pattern languages of programs, New York, NY, USA, pp 13:1–13:13

  12. Lanza M, D’Ambros M, Bacchelli A, Hattori L, Rigotti F (2013) Manhattan: supporting real-time visual team activity awareness. In: Proceedings of the 21st international conference on program comprehension, pp 207–210

  13. Levy M, Hazzan O (2009) Knowledge management in practice: the case of agile software development. IN: Proceedings of the 2009 ICSE workshops (CHASE’09), pp 60–65

  14. Maalej W, Thurimella AK (2015) Managing requirements knowledge. Springer, Berlin

    Google Scholar 

  15. Martin RC (2003) Agile software development: principles, patterns, and practices. Prentice Hall, Upper Saddle River

    Google Scholar 

  16. Mendes TS, de F. Farias MA, Mendonça M, Soares HF, Kalinowski M, Spínola RO (2016) Impacts of agile requirements documentation debt on software projects: a retrospective study. In: Proceedings of the 31st annual ACM symposium on applied computing, pp 1290–1295

  17. Moe NB, Faegri TE, Cruzes DS, Faugstad JE (2016) Enabling knowledge sharing in agile virtual teams. In: Proceedings of the 11th international conference on global software engineering, pp 29–34

  18. NTT DATA. NTT DATA: Global IT Innovator. http://www.nttdata.com/global/en/. Accessed 10 Jan 2017

  19. NTT R&D. NTT Research & Development. http://www.ntt.co.jp/RD/e/index.html. Accessed 10 Jan 2017

  20. Palmer SR, Felsing JM (2002) A practical guide to feature driven development. Prentice Hall, Upper Saddle River

    Google Scholar 

  21. Redmine. Flexible project management. https://redmine.org/. Accessed 03 Jan 2017

  22. Rubin KS (2012) Essential Scrum: a practical guide to the most common agile process. Addison-Wesley Professional, Boston

    Google Scholar 

  23. Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. EmpirSoftw Eng 14(2):131–165

    Google Scholar 

  24. Saito S, Iimura Y, Takahashi K, Massey AK, Antón AI (2014) Tracking Requirements evolution by using issue tickets: a case study of a document management and approval system. In: 36th international conference on software engineering, pp 245–254

  25. Saito S, Iimura Y, Tashiro H, Massey AK, Antón AI (2016) Visualizing the effects of requirements evolution. In: 38th International conference on software engineering, pp 152–161

  26. Saito S, Iimura Y, Massey AK, Antón AI (2017) How much undocumented knowledge is there in agile software development?: Case study on industrial project using issue tracking system and version control system. In: Proceedings of the 25th international conference on requirements engineering, pp 194–203

  27. Schwaber K, Beedle M (2001) Agile software development with Scrum. Prentice Hall, Upper Saddle River

    Google Scholar 

  28. Stettina CJ, Heijstek W (2011) Necessary and neglected?: An empirical study of internal documentation in agile software development teams. In: Proceedings of the 29th ACM international conference on design of communication, pp 159–166

  29. Trac. Integrated SCM & project management. https://trac.edgewall.org/. Accessed 03 Jan 2017

  30. Thurimella AK, Bruegge B, Janzen D (2017) Variability Plug-Ins for Requirements Tools: a Case-Based Theory Building Approach. IEEE Syst J 11(4):1935–1946

    Article  Google Scholar 

  31. Thurimella AK, Schubanz M, Pleuss A, Botterweck G (2017) Guidelines for managing requirement rationales. IEEE Softw 34(1):82–90

    Article  Google Scholar 

  32. Voigt S, von Garrel J, Müller J, Wirth D (2016) A study of documentation in agile software projects. In: Proceedings of the 10th ACM/IEEE international symposium on empirical software engineering and measurement, pp 4:1–4:6

  33. Wnuk K, Regnell B, Karlsson L (2009) Feature transition charts for visualization of cross-project scope evolution in large-scale requirements engineering for product lines. In: Proceedings of the 4th international workshop on requirements engineering visualization, pp 11–20

  34. Yin RK (2003) Case study research: design and methods, vol 5, 3rd edn. Sage, Thousand Oaks

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to Messrs. Takashi Hoshino, Keiichiro Horikawa, Daisuke Hamuro, and Masayuki Inoue of NTT, Tetsuo Kobashi, Masatoshi Hiraoka and Hironori Shibayama of NTT DATA, Dr. Noriaki Izumi and Motoi Yamane at Piecemeal Technology for their assistance in the case study. We also wish to thank the anonymous referees of both the Requirements Engineering Conference and this special issue of the Requirements Engineering Journal for their feedback.

Author information

Authors and Affiliations

  1. Software Innovation Center, NTT Corporation, Tokyo, Japan

    Shinobu Saito & Yukako Iimura

  2. Department of Information Systems, University of Maryland, Baltimore County, Baltimore, MD, USA

    Aaron K. Massey

  3. Georgia Institute of Technology School of Interactive Computing, Atlanta, GA, USA

    Annie I. Antón

Authors
  1. Shinobu Saito
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yukako Iimura
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aaron K. Massey
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Annie I. Antón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aaron K. Massey.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saito, S., Iimura, Y., Massey, A.K. et al. Discovering undocumented knowledge through visualization of agile software development activities. Requirements Eng 23, 381–399 (2018). https://doi.org/10.1007/s00766-018-0291-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-018-0291-4

Keywords

Search

Navigation