research-article

BugCache for inspections: hit or miss?

Authors:

Premkumar DevanbuAuthors Info & Claims

ESEC/FSE '11: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering

Pages 322 - 331

https://doi.org/10.1145/2025113.2025157

Published: 09 September 2011 Publication History

Abstract

Inspection is a highly effective but costly technique for quality control. Most companies do not have the resources to inspect all the code; thus accurate defect prediction can help focus available inspection resources. BugCache is a simple, elegant, award-winning prediction scheme that "caches" files that are likely to contain defects [12]. In this paper, we evaluate the utility of BugCache as a tool for focusing inspection, we examine the assumptions underlying BugCache with the aim of improving it, and finally we compare it with a simple, standard bug-prediction technique. We find that BugCache is, in fact, useful for focusing inspection effort; but surprisingly, we find that its performance, when used for inspections, is not much better than a naive prediction model -- viz., a model that orders files in the system by their count of closed bugs and chooses enough files to capture 20% of the lines in the system.

References

[1]

The economic impacts of inadequate infrastructure for software testing, \\www.nist.gov/director/planning/upload/report02-3.pdf.

[2]

E. Arisholm, L. Briand, and M. Fuglerud. Data mining techniques for building fault-proneness models in telecom java software. 2007.

[3]

E. Arisholm, L. C. Briand, and E. B. Johannessen. A systematic and comprehensive investigation of methods to build and evaluate fault prediction models. Journal of Systems and Software, 83(1):2--17, 2010.

Digital Library

[4]

A. Bachmann and A. Bernstein. Software process data quality and characteristics: a historical view on open and closed source projects. In IWPSE-Evol '09, 2009.

Digital Library

[5]

C. Bird, A. Bachmann, E. Aune, J. Duffy, A. Bernstein, V. Filkov, and P. Devanbu. Fair and balanced?: bias in bug-fix datasets. In Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, pages 121--130. ACM, 2009.

Digital Library

[6]

C. Catal and B. Diri. A systematic review of software fault prediction studies. Expert Systems with Applications, 36(4):7346--7354, 2009.

Digital Library

[7]

E. Engström, P. Runeson, and G. Wikstrand. An empirical evaluation of regression testing based on fix-cache recommendations. In Software Testing, Verification and Validation (ICST), 2010 Third International Conference on, pages 75--78. IEEE, 2010.

Digital Library

[8]

M. Fischer, M. Pinzger, and H. Gall. Populating a release history database from version control and bug tracking systems. In ICSM '03: Proceedings of the International Conference on Software Maintenance, pages 23, Washington, DC, USA, 2003. IEEE Computer Society.

Digital Library

[9]

A. E. Hassan. Predicting faults using the complexity of code changes. In Proceedings of the 31st International Conference on Software Engineering, ICSE '09, pages 78--88, Washington, DC, USA, 2009. IEEE Computer Society.

Digital Library

[10]

A. E. Hassan and R. C. Holt. The top ten list: Dynamic fault prediction. In Proceedings of the 21st IEEE International Conference on Software Maintenance, pages 263--272, Washington, DC, USA, 2005. IEEE Computer Society.

Digital Library

[11]

S. Kim, T. Zimmermann, K. Pan, and J. Jr. Automatic identification of bug-introducing changes. In ASE '06: Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering, pages 81--90, Washington, DC, USA, 2006. IEEE Computer Society.

Digital Library

[12]

S. Kim, T. Zimmermann, E. Whitehead Jr, and A. Zeller. Predicting faults from cached history. In Proceedings of the 29th international conference on Software Engineering, pages 489--498. IEEE Computer Society, 2007.

Digital Library

[13]

W. Kruskal and W. Wallis. Use of ranks in one-criterion variance analysis. Journal of the American statistical Association, 47(260):583--621, 1952.

[14]

J. Lawless and J. Lawless. Statistical models and methods for lifetime data. 1982.

[15]

A. Meneely and L. Williams. Secure open source collaboration: an empirical study of linus' law. In Proceedings of the 16th ACM conference on Computer and communications security, pages 453--462. ACM, 2009.

Digital Library

[16]

A. Mockus and L. Votta. Identifying reasons for software changes using historic databases. In Software Maintenance, 2000. Proceedings. International Conference on, pages 120--130. IEEE, 2002.

Digital Library

[17]

N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In Software Engineering, 2005. ICSE 2005. Proceedings. 27th International Conference on, pages 284--292. IEEE, 2005.

Digital Library

[18]

D. Perry, A. Porter, and L. Votta. Empirical studies of software engineering: a roadmap. In Proceedings of the conference on The future of Software engineering, pages 345--355. ACM New York, NY, USA, 2000.

Digital Library

[19]

C. Sadowski, C. Lewis, Z. Lin, Z. X., and E. J. Whitehead. An empirical analysis of the fixcache algorithm. In Proceedings of the Eigth MSR Conference (to appear), 2011.

Digital Library

[20]

J. Śliwerski, T. Zimmermann, and A. Zeller. When do changes induce fixes? In MSR '05: Proceedings of the 2005 international workshop on Mining software repositories, pages 1--5, New York, NY, USA, 2005. ACM.

Digital Library

[21]

D. Čubranić and G. C. Murphy. Hipikat: recommending pertinent software development artifacts. In ICSE '03: Proceedings of the 25th International Conference on Software Engineering, pages 408--418, Washington, DC, USA, 2003. IEEE Computer Society.

Digital Library

[22]

Z. Wang. Fix Cache Based Regression Test Selection. Master's thesis, Computer Science, Chalmers University of Technology, Sweden, 2010.

[23]

E. Weyuker, T. Ostrand, and R. Bell. Do too many cooks spoil the broth? Using the number of developers to enhance defect prediction models. Empirical Software Engineering, 13(5):539--559, 2008.

Digital Library

[24]

G. Wikstrand, R. Feldt, J. Gorantla, W. Zhe, and C. White. Dynamic regression test selection based on a file cache an industrial evaluation. In Software Testing Verification and Validation, 2009. ICST'09. International Conference on, pages 299--302. IEEE, 2009.

Digital Library

Cited By

Pei YSohn JHabchi SPapadakis M(2025)Non-Flaky and Nearly Optimal Time-Based Treatment of Asynchronous Wait Web TestsACM Transactions on Software Engineering and Methodology10.1145/369598934:2(1-29)Online publication date: 20-Jan-2025
https://dl.acm.org/doi/10.1145/3695989
Chowdhury SUddin GHemmati HHolmes R(2024)Method-Level Bug Prediction: Problems and PromisesACM Transactions on Software Engineering and Methodology10.1145/3640331Online publication date: 13-Jan-2024
https://doi.org/10.1145/3640331
Lyu YKang HWidyasari RLawall JLo D(2024)Evaluating SZZ Implementations: An Empirical Study on the Linux KernelIEEE Transactions on Software Engineering10.1109/TSE.2024.340671850:9(2219-2239)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3406718
Show More Cited By

Index Terms

BugCache for inspections: hit or miss?
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Process validation
        Walkthroughs

Recommendations

Recalling the "imprecision" of cross-project defect prediction
FSE '12: Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering

There has been a great deal of interest in defect prediction: using prediction models trained on historical data to help focus quality-control resources in ongoing development. Since most new projects don't have historical data, there is interest in ...
Checklist Based Reading's Influence on a Developer's Understanding
ASWEC '08: Proceedings of the 19th Australian Conference on Software Engineering

This paper addresses the influence the Checklist Based Reading inspection technique has on a developer's ability to modify inspected code. Traditionally, inspections have been used to detect defects within the development life cycle. This research ...
Investigating the Effect of Expert Ranking of Use Cases for Design Inspection
EUROMICRO '04: Proceedings of the 30th EUROMICRO Conference

Inspection is an important approach to reduce defects in software engineering artifacts. Reading techniques such as Usage-Based Reading (UBR) can focus inspector attention on specific types of defects (missing and wrong information) and specific defect ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ESEC/FSE '11: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering

September 2011

548 pages

ISBN:9781450304436

DOI:10.1145/2025113

General Chair:
Tibor Gyimóthy
University of Szeged, Hungary
,
Program Chair:
Andreas Zeller
Saarland University, Germany

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE'11

Sponsor:

SIGSOFT

ESEC/FSE'11: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering

September 5 - 9, 2011

Szeged, Hungary

Acceptance Rates

Overall Acceptance Rate 17 of 128 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

73
Total Citations
View Citations
927
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)14

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pei YSohn JHabchi SPapadakis M(2025)Non-Flaky and Nearly Optimal Time-Based Treatment of Asynchronous Wait Web TestsACM Transactions on Software Engineering and Methodology10.1145/369598934:2(1-29)Online publication date: 20-Jan-2025
https://dl.acm.org/doi/10.1145/3695989
Chowdhury SUddin GHemmati HHolmes R(2024)Method-Level Bug Prediction: Problems and PromisesACM Transactions on Software Engineering and Methodology10.1145/3640331Online publication date: 13-Jan-2024
https://doi.org/10.1145/3640331
Lyu YKang HWidyasari RLawall JLo D(2024)Evaluating SZZ Implementations: An Empirical Study on the Linux KernelIEEE Transactions on Software Engineering10.1109/TSE.2024.340671850:9(2219-2239)Online publication date: Sep-2024
https://doi.org/10.1109/TSE.2024.3406718
Niu FZhang EMayr-Dorn CAssunção WHuang LGe JLuo BEgyed A(2024)An extensive replication study of the ABLoTS approach for bug localizationEmpirical Software Engineering10.1007/s10664-024-10537-629:6Online publication date: 24-Aug-2024
https://doi.org/10.1007/s10664-024-10537-6
Rezaalipour MFuria C(2024)An empirical study of fault localization in Python programsEmpirical Software Engineering10.1007/s10664-024-10475-329:4Online publication date: 13-Jun-2024
https://dl.acm.org/doi/10.1007/s10664-024-10475-3
Wimalasooriya CLicorish Sda Costa DMacDonell S(2024)Just-in-Time crash prediction for mobile appsEmpirical Software Engineering10.1007/s10664-024-10455-729:3Online publication date: 8-May-2024
https://doi.org/10.1007/s10664-024-10455-7
Jing XChen HXu BJing XChen HXu B(2024)Other Research Questions of SDPIntelligent Software Defect Prediction10.1007/978-981-99-2842-2_7(171-201)Online publication date: 18-Jan-2024
https://doi.org/10.1007/978-981-99-2842-2_7
Niu FMayr-Dorn CAssunção WHuang LGe JLuo BEgyed A(2023)The ABLoTS Approach for Bug Localization: is it replicable and generalizable?2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR)10.1109/MSR59073.2023.00083(576-587)Online publication date: May-2023
https://doi.org/10.1109/MSR59073.2023.00083
Niu FAssunção WHuang LMayr-Dorn CGe JLuo BEgyed A(2023)RAT: A Refactoring-Aware Traceability Model for Bug Localization2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00028(196-207)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00028
Rosa GPascarella LScalabrino STufano RBavota GLanza MOliveto R(2023)A comprehensive evaluation of SZZ Variants through a developer-informed oracleJournal of Systems and Software10.1016/j.jss.2023.111729202(111729)Online publication date: Aug-2023
https://doi.org/10.1016/j.jss.2023.111729
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten