Skip to main content
SpringerLink
Log in

Studying the advancement in debugging practice of professional software developers

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

In 1997, Henry Lieberman stated that debugging is the dirty little secret of computer science. Since then, several promising debugging technologies have been developed such as back-in-time debuggers and automatic fault localization methods. However, the last study about the state-of-the-art in debugging is still more than 15 years old and so it is not clear whether these new approaches have been applied in practice or not. For that reason, we investigate the current state of debugging in a comprehensive study. First, we review the available literature and learn about current approaches and study results. Second, we observe several professional developers while debugging and interview them about their experiences. Third, we create a questionnaire that serves as the basis for a larger online debugging survey. Based on these results, we present new insights into debugging practice that help to suggest new directions for future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. The Pearson coefficient can be used to find linear relations between numeric variables. Values between \(-1\) and 1 indicate how well the actual relation can be approximated by a linear function (Pearson 1895).

  2. As early results of the survey have shown a large number of parallel behavioral bugs, we added it as a new choice that was not represented in Eisenstadt’s original version.

  3. A χ 2 test can be used for any kind of discretely categorized data with a large number of samples in each category. A large value indicates dependence and a low independence (Pearson 1900).

References

  • Agans, D. J. (2002). Debugging: The 9 indispensable rules for finding even the most elusive software and hardware problems. AMACOM Div American Mgmt Assn.

  • Agrawal, H., Horgan, J., London, S., & Wong, W. (1995). Fault localization using execution slices and dataflow tests. In conference on software reliability engineering (pp. 143–151).

  • Ahmadzadeh, M., Elliman, D., & Higgins, C. (2005). An analysis of patterns of debugging among novice computer science students. ACM SIGCSE Bulletin, 37(3), 84–88.

    Article  Google Scholar 

  • Artzi, S., Dolby, J., Tip, F., & Pistoia, M. (2010). Practical fault localization for dynamic web applications. In international conference on software engineering (pp. 265–274). ACM.

  • Arumuga Nainar, P., & Liblit, B. (2010). Adaptive bug isolation. In international conference on software engineering (pp. 255–264). ACM.

  • Baah, G. K., Podgurski, A., & Harrold, M. J. (2010). Causal inference for statistical fault localization. In international symposium on software testing and analysis (pp. 73–84). ACM.

  • Ballou, M. C. (2008). Improving software quality to drive business agility. IDC Survey and White Paper.

  • Chmiel, R., & Loui, M. C. (2004). Debugging: From novice to expert. ACM SIGCSE Bulletin, 36(1), 17–21.

    Article  Google Scholar 

  • Cleve, H., & Zeller, A. (2005). Locating causes of program failures. In international conference on software engineering (pp. 342–351). ACM.

  • Dallmeier, V., Lindig, C., & Zeller, A. (2005). Lightweight defect localization for java. In European conference on object-oriented programming (pp. 528–550). Springer.

  • Eisenstadt, M. (1997). My hairiest bug war stories. Communications of the ACM, 40(4), 30–37.

    Article  Google Scholar 

  • Gould, J. D. (1975). Some psychological evidence on how people debug computer programs. International Journal of Man-Machine Studies, 7(2), 151–182.

    Article  MATH  Google Scholar 

  • Gould, J. D., & Drongowski, P. (1974). An exploratory study of computer program debugging. The Journal of the Human Factors and Ergonomics Society, 16(3), 258–277.

    Article  Google Scholar 

  • Grötker, T., Holtmann, U., Keding, H., & Wloka, M. (2012). The developer’s guide to debugging (2nd ed.). NewYork: Self-publishing company.

    Google Scholar 

  • Gupta, N., He, H., Zhang, X., & Gupta, R. (2005). Locating faulty code using failure-inducing chops. In international conference on automated software engineering (pp. 263–272). ACM.

  • Hailpern, B., & Santhanam, P. (2002). Software debugging, testing, and verification. IBM Systems Journal, 41(1), 4–12.

    Article  Google Scholar 

  • Hanks, B., & Brandt, M. (2009). Successful and unsuccessful problem solving approaches of novice programmers. ACM SIGCSE Bulletin, 41(1), 24–28.

    Article  Google Scholar 

  • James, S., Bidgoli, M., & Hansen, J. (2008). Why sally and joey can’t debug: Next generation tools and the perils they pose. Journal of Computing Sciences in Colleges, 24(1), 27–35.

    Google Scholar 

  • Janssen, T., Abreu, R., & van Gemund, A. J. (2009). Zoltar: A toolset for automatic fault localization. In international conference on automated software engineering (pp. 662–664). IEEE Computer Society.

  • Jeffrey, D., Gupta, N., & Gupta, R. (2008). Fault localization using value replacement. In international symposium on software testing and analysis (pp. 167–178). ACM.

  • Jiang, L., & Su, Z. (2007). Context-aware statistical debugging: From bug predictors to faulty control flow paths. In international conference on automated software engineering (pp. 184–193). ACM.

  • Jones, J. A., Bowring, J. F., & Harrold, M. J. (2007). Debugging in parallel. In international symposium on software testing and analysis (pp. 16–26). ACM.

  • Kernighan, B. W., & Plauger, P. J. (1978). The elements of programming style (Vol. 1). NewYork: McGraw-Hill.

    MATH  Google Scholar 

  • LaToza, T. D., & Myers, B. A. (2010). Developers ask reachability questions. In international conference on software engineering, vol. 1, (pp. 185–194). IEEE.

  • Lencevicius, R. (2000). On-the-fly query-based debugging with examples. arXiv.

  • Lewis, B. (2003). Debugging backwards in time. In proceedings of the international workshop on automated debugging, AADEBUG (pp. 225–235). Arxiv.

  • Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005). Scalable statistical bug isolation. ACM SIGPLAN Notices, 40(6), 15–26.

    Article  Google Scholar 

  • Lieberman, H. (1997). The debugging scandal and what to do about it (introduction to the special section). Community ACM, 40(4), 26–29.

    Article  Google Scholar 

  • Liu, C., Yan, X., Fei, L., Han, J., & Midkiff, S. P. (2005). Sober: Statistical model-based bug localization. ACM SIGSOFT Software Engineering Notes, 30(5), 286–295.

    Article  Google Scholar 

  • Metzger, R. C. (2004). Debugging by thinking: A multidisciplinary approach. NewYork: Elsevier Digital Press.

    Google Scholar 

  • Murphy, L., Lewandowski, G., McCauley, R., Simon, B., Thomas, L., & Zander, C. (2008). Debugging: The good, the bad, and the quirky-a qualitative analysis of novices’ strategies. ACM SIGCSE Bulletin, 40(1), 163–167.

    Article  Google Scholar 

  • Park, S., Vuduc, R. W., & Harrold, M. J. (2010). Falcon: Fault localization in concurrent programs. In international conference on software engineering (pp. 245–254). ACM.

  • Pearson, K. (1895). Notes on regression and inheritance in the case of two parents. In proceedings of the royal society of London (pp. 240–242). The royal society.

  • Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. In philosophical magazine (pp. 157–175).

  • Perscheid, M. (2013). Test-driven fault navigation for debugging reproducible failures. Ph.D. thesis, Hasso Plattner Institute, University of Potsdam.

  • Renieres, M., & Reiss, S. P. (2003). Fault localization with nearest neighbor queries. In international conference on automated software engineering (pp. 30–39). IEEE.

  • Vessey, I. (1985). Expertise in debugging computer programs: A process analysis. International Journal of Man-Machine Studies, 23(5), 459–494.

    Article  Google Scholar 

  • Weiser, M. (1982). Programmers use slices when debugging. Communications of the ACM, 25(7), 446–452.

    Article  Google Scholar 

  • Wong, W. E., & Debroy, V. (2009). A survey of software fault localization. Department of Computer Science, University of Texas at Dallas, Technical Report UTDCS-45-09.

  • Yilmaz, C., Paradkar, A., & Williams, C. (2008). Time will tell: Fault localization using time spectra. In international conference on software engineering (pp. 81–90). ACM.

  • Zeller, A. (2002). Isolating cause-effect chains from computer programs. In symposium on foundations of software engineering (pp. 1–10). ACM.

  • Zeller, A. (2009). Why programs fail: A guide to systematic debugging. Burlington: Morgan Kaufmann.

    Google Scholar 

  • Zhang, X., Gupta, N., & Gupta, R. (2006). Locating faults through automated predicate switching. In international conference on software engineering (pp. 272–281). ACM.

Download references

Author information

Authors and Affiliations

  1. SAP Innovation Center, Potsdam, Germany

    Michael Perscheid

  2. Hasso Plattner Institute, University of Potsdam, Potsdam, Germany

    Benjamin Siegmund, Marcel Taeumel & Robert Hirschfeld

Authors
  1. Michael Perscheid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin Siegmund
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcel Taeumel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Robert Hirschfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Perscheid.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Perscheid, M., Siegmund, B., Taeumel, M. et al. Studying the advancement in debugging practice of professional software developers. Software Qual J 25, 83–110 (2017). https://doi.org/10.1007/s11219-015-9294-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-015-9294-2

Keywords

Search

Navigation