skip to main content
research-article

DARWIN: An approach to debugging evolving programs

Published: 03 July 2012 Publication History

Abstract

Bugs in programs are often introduced when programs evolve from a stable version to a new version. In this article, we propose a new approach called DARWIN for automatically finding potential root causes of such bugs. Given two programs—a reference program and a modified program—and an input that fails on the modified program, our approach uses symbolic execution to automatically synthesize a new input that (a) is very similar to the failing input and (b) does not fail. We find the potential cause(s) of failure by comparing control-flow behavior of the passing and failing inputs and identifying code fragments where the control flows diverge.
A notable feature of our approach is that it handles hard-to-explain bugs, like code missing errors, by pointing to code in the reference program. We have implemented this approach and conducted experiments using several real-world applications, such as the Apache Web server, libPNG (a library for manipulating PNG images), and TCPflow (a program for displaying data sent through TCP connections). In each of these applications, DARWIN was able to localize bugs with high accuracy. Even though these applications contain several thousands of lines of code, DARWIN could usually narrow down the potential root cause(s) to less than ten lines. In addition, we find that the inputs synthesized by DARWIN provide additional value by revealing other undiscovered errors.

References

[1]
Agrawal, H. and Horgan, J. R. 1990. Dynamic program slicing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'90). ACM Press, New York, NY, 246--256.
[2]
Apache. 2009. Apache Web server. http://httpd.apache.org/.
[3]
Apiwattanapong, T., Orso, A., and Harrold, M. 2004. A differencing algorithm for object-oriented programs. In Proceedings of the International Conference on Automated Software Engineering (ASE). IEEE Computer Society, Los Alamitos, CA.
[4]
Ball, T., Naik, M., and Rajamani, S. 2003. From symptom to cause: Localizing errors in counterexample traces. In Proceedings of the International Symposium on Principles of Programming Languages (POPL). ACM Press, New York, NY.
[5]
Barrett, C. and Tinelli, C. 2007. CVC3. In Proceedings of the 19th International Conference on Computer-Aided Verification. 298--302.
[6]
Brumley, D., Caballero, J., Liang, Z., Newsome, J., and Song, D. 2007. Towards automatic discovery of deviations in binary implementations with applications to error detection and fingerprint generation. In Proceedings of the USENIX Security Conference. USENIX Association, Berkeley, CA.
[7]
Brummayer, R. and Biere, A. 2009. Boolector: An efficient smt solver for bit-vectors and arrays. In Proceedings of the 15th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'09). 174--177.
[8]
Bruttomesso, R., Cimatti, A., Franzén, A., Griggio, A., and Sebastiani, R. 2008. The MathSAT 4 SMT Solver. In Proceedings of the International Conference on Computer Aided Verification. 299--303.
[9]
Chen, Y., Rosenblum, D., and Vo, K. 1994. Testtube: A system for selective regression testing. In Proceedings of the International Conference on Software Engineering. IEEE Computer Society Press, Los Alamitos, CA.
[10]
Csallner, C. and Smaragdakis, Y. 2006. DSD-Crasher: A hybrid analysis tool for bug finding. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA). ACM Press, New York, NY.
[11]
de Moura, L. and Bjorner, N. 2008. Z3: An efficient SMT solver. In Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS).
[12]
Elbaum, S., Malishevsky, A., and Rothermel, G. 2000. Prioritizing test cases for regression testing. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA). ACM Press, New York, NY.
[13]
Ganesh, V. and Dill, D. L. 2007. A decision procedure for bit-vectors and arrays. In Proceedings of the Computer Aided Verification Conference (CAV). 524--536. Available online at http://sites.google.com/site/stpfastprover/.
[14]
Giroux, O. and Robillard, M. P. 2006. Detecting increases in feature coupling using regression tests. In Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT'06/FSE-14). ACM Press, New York, NY, 163--174.
[15]
Godefroid, P., Klarlund, N., and Sen, K. 2005. DART: Directed automated random testing. In Proceedings of the Conference on Programming Languages Design and Implementation (PLDI). ACM Press, New York. NY.
[16]
Guo, L., Roychoudhury, A., and Wang, T. 2006. Accurately choosing execution runs for software fault localization. In Proceedings of the International Conference on Compiler Construction (CC).
[17]
Horowitz, S. 1990. Identifying the semantic and textual differences between two versions of a program. In Proceedings of the International Conference on Programming Language Design and Implementation (PLDI). ACM Press, New York, NY.
[18]
Hovemeyer, D. and Pugh, W. 2004. Finding bugs is easy. In Proceedings of the Companion to the 19th Annual ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA'04). ACM Press, New York, NY, 132--136.
[19]
Huang, S. 2009. Miniweb Web server. http://miniweb.sourceforge.net/.
[20]
Jackson, D. and Ladd, D. A. 1994. Semantic diff: A tool for summarizing the effects of modifications. In Proceedings of the International Conference on Software Maintenance. 243--252.
[21]
Korel, B. and Laski, J. W. 1988. Dynamic program slicing. Inform. Process. Letters 29, 3, 155--163.
[22]
Liblit, B. 2005. Cooperative bug isolation. Ph.D. dissertation, UC Berkeley.
[23]
Liblit, B., Naik, M., Zheng, A., Aiken, A., and Jordan, M. 2005. Scalable statistical bug isolation. In Proceedings of the Conference on Programming Language Design and Implementation (PLDI). ACM Press, New York, NY.
[24]
LibPNG. 2009. libPNG library. http://www.libpng.org.
[25]
Person, S., Dwyer, M., Elbaum, S., and Pasareanu, C. 2008. Differential symbolic execution. In Proceedings of the International Conference on Foundations of Software Engineering (FSE). ACM Press, New York, NY.
[26]
QEMU. 2009. QEMU emulator. http://www.qemu.org.
[27]
Qi, D., Roychoudhury, A., Liang, Z., and Vaswani, K. 2009. Darwin: An approach for debugging evolving programs. In Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC-FSE). ACM Press, New York, NY, 33--42.
[28]
Ranise, S. and Tinelli, C. 2003. The SMT-LIB format: An initial proposal. In Proceedings of the Workshop on Pragmatics of Decision Procedures in Automated Reasoning (PDPAR).
[29]
Ren, X., Shah, F., Tip, F., Ryder, B. G., and Chesley, O. 2004. Chianti: A tool for change impact analysis of java programs. In Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA'04). ACM Press, New York, NY, 432--448.
[30]
Renieris, M. and Reiss, S. P. 2003. Fault localization with nearest neighbor queries. In Proceedings of the International Conference on Automated Software Engineering (ASE). IEEE Computer Society, Los Alamitos, CA.
[31]
Rothermel, G. and Harrold, M. J. 1997. A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6, 2, 173--210.
[32]
Santelices, R., Chittimalli, P., Apiwattanapong, T., Orso, A., and Harrold, M. 2008. Test-suite augmentation for evolving software. In Proceedings of the International Conference on Automated Software Engineering (ASE). IEEE Computer Society, Los Alamitos, CA.
[33]
Savant. 2009. Savant Web server. http://savant.sourceforge.net/info.html.
[34]
Seacord, R., Plakosh, D., and Lewis, G. 2003. Modernizing Legacy Systems: Software Technologies, Engineering Processes, and Business Practices. Addison-Wesley, Boston, MA.
[35]
Sen, K., Marinov, D., and Agha, G. 2005. Cute: A concolic unit testing engine for c. In Proceedings of the 10th European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM Press, New York, NY, 263--272.
[36]
Sillito, J., Murphy, G., and De Volder, K. 2006. Questions programmers ask during software evolution tasks. In Proceedings of the International Conference on Foundations of Software Engineering (FSE). ACM Press, New York, NY.
[37]
Song, D., Brumley, D., Yin, H., Caballero, J., Jager, I., Kang, M. G., Liang, Z., Newsome, J., Poosankam, P., and Saxena, P. 2008. BitBlaze: A new approach to computer security via binary analysis. In Proceedings of the 4th International Conference on Information Systems Security. Keynote invited paper.
[38]
Sridharan, M., Fink, S. J., and Bodik, R. 2007. Thin slicing. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'07). ACM Press, New York, NY, 112--122.
[39]
Srivastava, A. and Thiagarajan, J. 2002. Effectively prioritizing tests in development environment. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). ACM Press, New York, NY, 97--106.
[40]
Wang, T. and Roychoudhury, A. 2004. Using compressed bytecode traces for slicing Java programs. In Proceedings of the 26th International Conference on Software Engineering (ICSE). IEEE Computer Society, Los Alamitos, CA, 512--521.
[41]
Zeller, A. 1999. Yesterday, my program worked. Today, it does not. Why? In Proceedings of the 7th European Software Engineering Conference held jointly with the ACM SIGSOFT International Symposium on Foundations of Software Engineering. 253--267.
[42]
Zeller, A. 2002. Isolating cause-effect chains from computer programs. In Proceedings of the 10th ACM SIGSOFT Symposium on Foundations of Software Engineering. ACM Press, New York, NY, 1--10.
[43]
Zeller, A. and Hildebrandt, R. 2002. Simplifying and isolating failure-inducing input. IEEE Trans. Softw. Eng. 28, 2, 183--200.
[44]
Zhang, X., Gupta, N., and Gupta, R. 2006. Pruning dynamic slices with confidence. In Proceedings of the International Conference on Programming Language Design and Implementation (PLDI). ACM Press, New York, NY, 169--180.
[45]
Zhang, X., Tallam, S., Gupta, N., and Gupta, R. 2007. Towards locating execution omission errors. In Proceedings of the International Conference on Programming Language Design and Implementation (PLDI). ACM Press, New York, NY, 415--424.

Cited By

View all
  • (2023)TransMap: Pinpointing Mistakes in Neural Code TranslationProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616322(999-1011)Online publication date: 30-Nov-2023
  • (2023)Responsibility in Context: On Applicability of Slicing in Semantic Regression AnalysisProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00057(563-575)Online publication date: 14-May-2023
  • (2021)Explaining Regressions via Alignment Slicing and MendingIEEE Transactions on Software Engineering10.1109/TSE.2019.294956847:11(2421-2437)Online publication date: 1-Nov-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Software Engineering and Methodology
ACM Transactions on Software Engineering and Methodology  Volume 21, Issue 3
June 2012
239 pages
ISSN:1049-331X
EISSN:1557-7392
DOI:10.1145/2211616
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 July 2012
Accepted: 01 February 2011
Revised: 01 May 2010
Received: 01 October 2009
Published in TOSEM Volume 21, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Software debugging
  2. software evolution
  3. symbolic execution

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)24
  • Downloads (Last 6 weeks)0
Reflects downloads up to 30 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)TransMap: Pinpointing Mistakes in Neural Code TranslationProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616322(999-1011)Online publication date: 30-Nov-2023
  • (2023)Responsibility in Context: On Applicability of Slicing in Semantic Regression AnalysisProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00057(563-575)Online publication date: 14-May-2023
  • (2021)Explaining Regressions via Alignment Slicing and MendingIEEE Transactions on Software Engineering10.1109/TSE.2019.294956847:11(2421-2437)Online publication date: 1-Nov-2021
  • (2020)Causal testingProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380377(87-99)Online publication date: 27-Jun-2020
  • (2020)Improving Fault-Localization Accuracy by Referencing Debugging History to Alleviate Structure Bias in Code SuspiciousnessIEEE Transactions on Reliability10.1109/TR.2020.298297569:3(1021-1049)Online publication date: Sep-2020
  • (2020)Test Case Understandability ModelIEEE Access10.1109/ACCESS.2020.30228768(169036-169046)Online publication date: 2020
  • (2019)Dimensions of Robust Security Testing in Global Software EngineeringHuman Factors in Global Software Engineering10.4018/978-1-5225-9448-2.ch010(252-272)Online publication date: 2019
  • (2018)Behaviour Preservation across Code Versions in ErlangScientific Programming10.1155/2018/92517622018Online publication date: 13-Jun-2018
  • (2018)Symbolic execution with existential second-order constraintsProceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3236024.3236049(389-399)Online publication date: 26-Oct-2018
  • (2017)Dependence Guided Symbolic ExecutionIEEE Transactions on Software Engineering10.1109/TSE.2016.258406343:3(252-271)Online publication date: 1-Mar-2017
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media