Skip to main content
SpringerLink
Log in

On the documentation of self-admitted technical debt in issues

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Self-Admitted Technical Debt (Satd) is a particular case of Technical Debt (TD) in which developers rely on source code comments (Satd-C) or labeled issues (Satd-I) to report their sub-optimal technical solutions. In this paper, we first explore a sample of 286 Satd-I instances collected from five open source projects, including Microsoft Visual Studio Code and GitLab Community Edition. We show that in 45% of the studied issues TD was introduced to ship earlier (i.e., to deliver faster), and in almost 60% it refers to Design flaws. Besides, we report that most developers pay Satd-I to reduce its costs or interests (66%). To complement the previous exploratory results, we investigate the adoption of tools to support Satd-I documentation. For that, we build a large-scale dataset of 72K Satd-C and 20K Satd-I instances, extracted from 190 GitHub projects. We also implement a prototype tool, called AdmiTD, to automatically report Satd-C as GitHub issues. We use this dataset and tool to reveal that developers are not interested in the automatic transformation of Satd-C in Satd-I. Moreover, we show that it might not be feasible to create a tool to recommend explicit links between Satd-C and Satd-I instances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Code Availability

https://doi.org/10.5281/zenodo.6532378

Notes

  1. In fact, Satd can also be documented in other artifacts, such as commit messages, wikis, forum discussions, etc.

  2. https://gitlab.com/gitlab-org/gitlab-ce/issues/34659

  3. https://gitlab.com/gitlab-org/gitlab-ce/issues/44726

  4. We acknowledge that some of the non-selected issues might also refer to TD (for example, developers may have forgotten to label them as such). Despite that, this fact does not invalidate our key goal in the paper, which is studying a valid and large sample of issues that explicitly document TD.

  5. Although this taxonomy is based on research published before the Satd term was coined, we decided to use it because it covers problems that can occur beyond the source code (e.g., UI and Build problems).

  6. It is worth mentioning that 33 developers (out of the 85 contacted) were also responsible for opening the corresponding issues. Among the 30 developers who answered our emails, 10 were also the authors of the Satd-I.

  7. https://news.ycombinator.com/item?id=22915584

  8. We acknowledge that some of the selected issues might not document TD concerns (for example, developers may have incorrectly labeled some issues). Despite that, this fact does not invalidate our key goal in this extended study, which is studying tool support for Satd documentation.

  9. https://pypi.org/project/comment-parser/

  10. However, we highlight that it is not possible to directly compare and contrast the two sets (Satd-C and Satd-I), since the metrics used in each one are distinct.

  11. https://github.com/admitd

  12. https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.todo-tree

  13. Interestingly, this comment suggests that previous studies on Satd-C might have maximized the occurrence of TD, by considering any TODO to be problematic. As stated by the respondent, a TODO can just be a minor observation for the developer in the future.

  14. https://docs.python.org/3/library/difflib.html#module-difflib

  15. Determined after specifying a limit of 95% confidence level, with a margin of error of 10%.

References

  • Alves N S, Mendes T S, de Mendonça M G, Spínola RO, Shull F, Seaman C (2016) Identification and management of technical debt. Inf Softw Technol 70(C):100–121

    Article  Google Scholar 

  • Azuma H, Matsumoto S, Kamei Y, Kusumoto S (2022) An empirical study on self-admitted technical debt in dockerfiles. Empir Softw Eng 27:1–26

    Article  Google Scholar 

  • Bavota G, Russo B (2016) A large-scale empirical study on self-admitted technical debt. In: 13th Working conference on mining software repositories (MSR), pp 315–326

  • Bellomo S, Nord R L, Ozkaya I, Popeck M (2016) Got technical debt? Surfacing elusive technical debt in issue trackers. In: 13th International conference on mining software repositories (MSR), pp 327–338

  • Borges H, Hora A, Valente M T (2016) Understanding the factors that impact the popularity of GitHub repositories. In: 32nd IEEE international conference on software maintenance and evolution (ICSME), pp 334–344

  • Cabot J, Cánovas Izquierdo J L, Cosentino V, Rolandi B (2015) Exploring the use of labels to categorize issues in open-source software projects. In: 22nd International conference on software analysis, evolution, and reengineering (SANER), pp 550–554

  • Cunningham W (1992) The WyCash portfolio management system. In: 7th Object-oriented programming systems, languages, and applications (OOPSLA), pp 29–30

  • da Fonseca Lage L C, Kalinowski M, Trevisan D, Spinola R (2019) Usability technical debt in software projects: a multi-case study. In: 13th International symposium on empirical software engineering and measurement (ESEM), pp 1–6

  • Dai K, Kruchten P (2017) Detecting technical debt through issue trackers. In: 5th International workshop on quantitative approaches to software quality (quASoq), pp 59–65

  • de Freitas Farias M A, de Mendonça Neto M G, Kalinowski M, Spínola R O (2020) Identifying self-admitted technical debt through code comment analysis with a contextualized vocabulary. Inf Softw Technol 121:106270–106270

    Article  Google Scholar 

  • de Lima B S, Garcia R E, Eler D M (2022) Toward prioritization of self-admitted technical debt: an approach to support decision to payment. Softw Qual J 1:1–1

    Google Scholar 

  • Ernst N A, Bellomo S, Ozkaya I, Nord R L, Gorton I (2015) Measure it? Manage it? Ignore it? Software practitioners and technical debt. In: 10th Joint meeting on foundations of software engineering (FSE), pp 50–60

  • Fahid F M, Yu Z, Menzies T (2019) Better technical debt detection via SURVEYing. Computing Research Repository. arXiv:1905.08297

  • Farias M A, Santos J A, Kalinowski M, Mendonça M, Spinola R O (2016) Investigating the identification of technical debt through code comment analysis. In: 18th International conference on enterprise information systems (ICEIS), pp 284–309

  • Flisar J, Podgorelec V (2019) Identification of self-admitted technical debt using enhanced feature selection based on word embedding. IEEE Access 7(1):106475–106494

    Article  Google Scholar 

  • Fowler M (2019) Technicaldebtquadrant. https://martinfowler.com/bliki/TechnicalDebtQuadrant.html. Accessed 10 Oct 2019

  • Fucci G, Zampetti F, Serebrenik A, Penta M D (2020) Who (self) admits technical debt?. In: 36th International conference on software maintenance and evolution (ICSME), pp 672–676

  • Fucci G, Cassee N W, Zampetti F, Novielli N, Serebrenik A, Penta M D (2021) Waiting around or job half-done? Sentiment in self-admitted technical debt. In: 18th International conference on mining software repositories (MSR), pp 1–10

  • Guo Z, Liu S, Liu J, Li Y, Chen L, Lu H, Zhou Y (2021) How far have we progressed in identifying self-admitted technical debts? A comprehensive empirical study. ACM Trans Softw Eng Methodol 30:1–56

    Article  Google Scholar 

  • Huang Q, Shihab E, Xia X, Lo D, Li S (2018) Identifying self-admitted technical debt in open source projects using text mining. Empir Softw Eng 23(1):418–451

    Article  Google Scholar 

  • Iammarino M, Zampetti F, Aversano L, Penta M D (2019) Self-admitted technical debt removal and refactoring actions: co-occurrence or more?. In: 35th International conference on software maintenance and evolution (ICSME), pp 186–190

  • Iammarino M, Zampetti F, Aversano L, Di Penta M (2021) An empirical study on the co-occurrence between refactoring actions and self-admitted technical debt removal. J Syst Softw 178:110976–110976

    Article  Google Scholar 

  • Kamei Y, Maldonado E D S, Shihab E, Ubayashi N (2016) Using analytics to quantify the interest of self-admitted technical debt. In: 1st International workshop on technical debt analytics (TDA), pp 68–71

  • Kashiwa Y, Nishikawa R, Kamei Y, Kondo M, Shihab E, Sato R, Ubayashi N (2022) An empirical study on self-admitted technical debt in modern code review. Inf Softw Technol 146:106855–106855

    Article  Google Scholar 

  • Kruchten P, Nord R L, Ozkaya I (2012) Technical debt: from metaphor to theory and practice. IEEE Softw 29(6):18–21

    Article  Google Scholar 

  • Li Z, Avgeriou P, Liang P (2015) A systematic mapping study on technical debt and its management. J Syst Softw 101(C):193–220

    Article  Google Scholar 

  • Li Y, Soliman O, Avgeriou P (2020) Identification and remediation of self-admitted technical debt in issue trackers. In: 46th Euromicro conference on software engineering and advanced applications (SEAA), pp 495–503

  • Li Y, Soliman M, Avgeriou P (2022) Identifying self-admitted technical debt in issue tracking systems using machine learning. Empir Softw Eng 1:1–1

    Google Scholar 

  • Lim E, Taksande N, Seaman C (2012) A balancing act: what software practitioners have to say about technical debt. IEEE Softw 29(6):22–27

    Article  Google Scholar 

  • Liu Z, Huang Q, Xia X, Shihab E, Lo D, Li S (2018) SATD detector: a text-mining-based self-admitted technical debt detection tool. In: 40th International conference on software engineering: companion (ICSE-companion), pp 9–12

  • Maipradit R, Lin B, Nagy C, Bavota G, Lanza M, Hata H, Matsumoto K (2020a) Automated identification of on-hold self-admitted technical debt. In: 20th International working conference on source code analysis and manipulation (SCAM), pp 54–64

  • Maipradit R, Treude C, Hata H, Matsumoto K (2020b) Wait for it: identifying “on-hold” self-admitted technical debt. Empir Softw Eng 25:3770–3798

    Article  Google Scholar 

  • Maldonado E D S, Shihab E (2015) Detecting and quantifying different types of self-admitted technical debt. In: 7th International workshop on managing technical debt (MTD), pp 9–15

  • Maldonado E D S, Abdalkareem R, Shihab E, Serebrenik A (2017a) An empirical study on the removal of self-admitted technical debt. In: 33rd International conference on software maintenance and evolution (ICSME), pp 238–248

  • Maldonado E D S, Shihab E, Tsantalis N (2017b) Using natural language processing to automatically detect self-admitted technical debt. IEEE Trans Softw Eng 43(11):1044–1062

    Article  Google Scholar 

  • Martini A, Besker T, Bosch J (2018) Technical debt tracking: current state of practice: a survey and multiple case study in 15 large organizations. Sci Comput Program 163:42–61

    Article  Google Scholar 

  • Potdar A, Shihab E (2014) An exploratory study on self-admitted technical debt. In: 30th International conference on software maintenance and evolution (ICSME), pp 91–100

  • Ren X, Xing Z, Xia X, Lo D, Wang X, Grundy J (2019) Neural network-based detection of self-admitted technical debt: from performance to explainability. ACM Trans Softw Eng Methodol 28(3):1–45

    Article  Google Scholar 

  • Rios N, de Mendonça Neto M G, Spínola R O (2018) A tertiary study on technical debt: types, management strategies, research trends, and base information for practitioners. Inf Softw Technol 102(1):117–145

    Article  Google Scholar 

  • Sierra G, Shihab E, Kamei Y (2019a) A survey of self-admitted technical debt. J Syst Softw 152(1):70–82

    Article  Google Scholar 

  • Sierra G, Tahmid A, Shihab E, Tsantalis N (2019b) Is self-admitted technical debt a good indicator of architectural divergences?. In: 26th International conference on software analysis, evolution and reengineering (SANER), pp 534–543

  • Silva H, Valente M T (2018) What’s in a GitHub star? Understanding repository starring practices in a social coding platform. J Syst Softw 146:112–129

    Article  Google Scholar 

  • Silva M, Terra R, Valente M T (2016) Does technical debt lead to the rejection of pull requests?. In: 12nd Brazilian symposium on information systems (SBSI), pp 1–7

  • Spencer D (2009) Card sorting: designing usable categories. Rosenfeld Media

  • Storey M A, Ryall J, Bull R I, Myers D, Singer J (2008) TODO or to bug: exploring how task annotations play a role in the work practices of software developers. In: 30th International conference on software engineering (ICSE), pp 251–260

  • Tan J, Feitosa D, Avgeriou P (2021) Do practitioners intentionally self-fix technical debt and why?. In: 37th International conference on software maintenance and evolution (ICSME), pp 251–262

  • Vidoni M (2021) Self-admitted technical debt in r packages: an exploratory study. In: 18th International conference on mining software repositories (MSR), pp 179–189

  • Wang X, Liu J, Li L, Chen X, Liu X, Wu H (2020) Detecting and explaining self-admitted technical debts with attention-based neural networks. In: 35th International conference on automated software engineering (ASE), pp 871–882

  • Wehaibi S, Shihab E, Guerrouj L (2016) Examining the impact of self-admitted technical debt on software quality. In: 23rd International conference on software analysis, evolution, and reengineering (SANER), pp 179–188

  • Wiese M, Rachow P, Riebisch M, Schwarze J (2022) Preventing technical debt with the tap framework for technical debt aware management. Inf Softw Technol 148:106926

    Article  Google Scholar 

  • Wohlin C, Runeson P, Hst M, Ohlsson M C, Regnell B, Wessln A (2012) Experimentation in software engineering. Springer

  • Xavier L, Ferreira F, Brito R, Valente M T (2020) Beyond the code: mining self-admitted technical debt in issue tracker systems. In: 17th International conference on mining software repositories (MSR), pp 137–146

  • Xavier L, Montandon J E, Valente M T (2022) Comments or issues: where to document technical debt? IEEE Softw 1:1–14

    Google Scholar 

  • Xiao T, Wang D, Mcintosh S, Hata H, Kula R G, Ishio T, Matsumoto K (2021) Characterizing and mitigating self-admitted technical debt in build systems. IEEE Trans Softw Eng 1:1–1

    Google Scholar 

  • Yli-Huumo J, Maglyas A, Smolander K (2016) How do software development teams manage technical debt?—an empirical study. J Syst Softw 120 (C):195–218

    Article  Google Scholar 

  • Zampetti F, Noiseux C, Antoniol G, Khomh F, Penta M D (2017) Recommending when design technical debt should be self-admitted. In: 33rd International conference on software maintenance and evolution (ICSME), pp 216–226

  • Zampetti F, Serebrenik A, Penta M D (2018) Was self-admitted technical debt removal a real removal? An in-depth perspective. In: 15th International conference on mining software repositories (MSR), pp 526–536

  • Zampetti F, Serebrenik A, Penta M D (2020) Automatically learning patterns for self-admitted technical debt removal. In: 27th International conference on software analysis, evolution and reengineering (SANER), pp 355–366

  • Zazworka N, Shaw M A, Shull F, Seaman C (2011) Investigating the impact of design debt on software quality. In: 2nd workshop on managing technical debt (MTD), pp 17–23

  • Zazworka N, Spínola RO, Vetro’ A, Shull F, Seaman C (2013) A case study on effectively identifying technical debt. In: 17th International conference on evaluation and assessment in software engineering (EASE), pp 42–47

Download references

Acknowledgements

We thank the developers who participated in our surveys and shared their ideas and practices about technical debt documentation and payment. This research is supported by grants from FAPEMIG, CNPq, and CAPES.

Funding

Not Applicable

Author information

Authors and Affiliations

  1. Department of Computer Science, UFMG, Belo Horizonte, Brazil

    Laerte Xavier, Rodrigo Brito & Marco Tulio Valente

  2. Technical College (COLTEC), UFMG, Belo Horizonte, Brazil

    João Eduardo Montandon

  3. Center of Informatics, IF SUDESTE MG, Barbacena, Brazil

    Fabio Ferreira

Authors
  1. Laerte Xavier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Eduardo Montandon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabio Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rodrigo Brito
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marco Tulio Valente
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laerte Xavier.

Ethics declarations

Conflicts of Interest/Competing Interests

Not Applicable

Additional information

Communicated by: Hideaki Hata

Availability of Data and Material

https://doi.org/10.5281/zenodo.6532378

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xavier, L., Montandon, J.E., Ferreira, F. et al. On the documentation of self-admitted technical debt in issues. Empir Software Eng 27, 163 (2022). https://doi.org/10.1007/s10664-022-10203-9

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-022-10203-9

Keywords

Search

Navigation