Masatomo Hashimoto
ABSTRACT
Delta debugging (DD) is an approach to automating the debugging activities based on systematic testing. DD algorithms find the cause of a regression of a program by minimizing the changes applied between a working version and a faulty version of the program. However, it is still an open problem to minimize a huge set of changes while avoiding any invalid subsets that do not result in testable programs, especially in case that no software configuration management system is available. In this paper, we propose a rule-based approach to syntactic and semantic decomposition of changes into independent components to facilitate DD on source code changes, and hence to extract patches automatically. For analyzing changes, we make use of tree differencing on abstract syntax trees instead of common differencing on plain texts. We have developed an experimental implementation for Java programs and applied it to 194 bug fixes from Defects4J and 8 real-life regression bugs from 6 open source Java projects. Compared to a DD tool based on plain text differencing, it extracted patches whose size is reduced by 50% at the cost of 5% more test executions for the former dataset and by 73% at the cost of 40% more test executions for the latter, both on average.
- Hiralal Agrawal and Joseph R. Horgan. 1990. Dynamic Program Slicing. In Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation (PLDI ’90). ACM, New York, NY, USA, 246–256. Google Scholar
Digital Library
- Anonymous. 2009. {HSQLDB-933} UPDATE statement with IN criterion recently broken. Retrieved August 4, 2018 from https://sourceforge.net/p/hsqldb/bugs/ 933/Google Scholar
- Cyrille Artho. 2011. Iterative Delta Debugging. International Journal on Software Tools for Technology Transfer 13, 3 (2011), 223–246. s10009-010-0139-9Google ScholarDigital Library
- Thomas Ball, Mayur Naik, and Sriram K. Rajamani. 2003. From Symptom to Cause: Localizing Errors in Counterexample Traces. In Proceedings of the 30th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’03). ACM, New York, NY, USA, 97–105. Google ScholarDigital Library
- Jean-Francois Bergeretti and Bernard A. Carré. 1985. Information-flow and Dataflow Analysis of While-programs. ACM Transactions on Programming Languages and Systems 7, 1 (Jan. 1985), 37–61. Google ScholarDigital Library
- Árpád Beszédes, Csaba Faragó, Zsolt Mihály Szabó, János Csirik, and Tibor Gyimóthy. 2002. Union Slices for Program Maintenance. In Proceedings of the 18th International Conference on Software Maintenance (ICSM ’02). IEEE Computer Society, Washington, DC, USA, 12–21. Google ScholarDigital Library
- Philip Bille. 2005. A survey on tree edit distance and related problems. Theoretical Computer Science 337, 1-3 (June 2005), 217–239. 2004.12.030 Google ScholarDigital Library
- François Billioud. 2017. {JSOUP-920} Regression: Wrong parsing for clippath. Retrieved August 4, 2018 from https://github.com/jhy/jsoup/issues/920Google Scholar
- Marcel Böhme, Ezekiel O. Soremekun, Sudipta Chattopadhyay, Emamurho Ugherughe, and Andreas Zeller. 2017. Where Is the Bug and How Is It Fixed? An Experiment with Practitioners. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE ’17). ACM, New York, NY, USA, 117–128. Google ScholarDigital Library
- Paul C. Bryan. 2008. JavaScript Object Notation (JSON) Patch. Retrieved August 4, 2018 from https://tools.ietf.org/html/rfc6902Google Scholar
- Jong-Deok Choi and Andreas Zeller. 2002. Isolating Failure-inducing Thread Schedules. In Proceedings oxof the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA ’02). ACM, New York, NY, USA, 210–220. Google ScholarDigital Library
- Holger Cleve and Andreas Zeller. 2005. Locating Causes of Program Failures. In Proceedings of the 27th International Conference on Software Engineering (ICSE ’05). ACM, New York, NY, USA, 342–351. Google ScholarDigital Library
- Zack Coker and Munawar Hafiz. 2013. Program Transformations to Fix C Integers. In Proceedings of the 35th International Conference on Software Engineering (ICSE ’13). IEEE Computer Society, Washington, DC, USA, 792–801. 1109/ICSE.2013.6606625 Google ScholarDigital Library
- Christian Couder. 2009. Fighting regressions with git bisect. Retrieved August 4, 2018 from https://www.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.Google Scholar
- htmlGoogle Scholar
- Benoit de Rancourt. 2012. {MATH-805} Percentile calculation is very slow when input data are constants. Retrieved August 4, 2018 from https://issues.apache. org/jira/browse/MATH-805Google Scholar
- Jeffrey Dean, David Grove, and Craig Chambers. 1995. Optimization of Object-Oriented Programs Using Static Class Hierarchy Analysis. In Proceedings of the 9th European Conference on Object-Oriented Programming (ECOOP ’95). Springer-Verlag, Berlin, Heidelberg, 77–101. Google ScholarDigital Library
- Vidroha Debroy and W. Eric Wong. 2010. Using Mutation to Automatically Suggest Fixes for Faulty Programs. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST ’10). IEEE Computer Society, Washington, DC, USA, 65–74. Google ScholarDigital Library
- Jean-Rémy Falleri, Floréal Morandat, Xavier Blanc, Matias Martinez, and Martin Monperrus. 2014. Fine-grained and Accurate Source Code Differencing. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering (ASE ’14). ACM, New York, NY, USA, 313–324. Google ScholarDigital Library
- Jeanne Ferrante, Karl J. Ottenstein, and Joe D. Warren. 1987. The Program Dependence Graph and Its Use in Optimization. ACM Transactions on Programming Languages and Systems 9, 3 (July 1987), 319–349. Google ScholarDigital Library
- Beat Fluri, Michael Würsch, Martin Pinzger, and Harald Gall. 2007. Change Distilling: Tree Differencing for Fine-Grained Source Code Change Extraction. IEEE Transactions on Software Engineering 33, 11 (2007), 725–743. org/10.1109/TSE.2007.70731 Google ScholarDigital Library
- Python Software Foundation. 2001. difflib: helpers for computing deltas. https: //docs.python.org/2/library/difflib.htmlGoogle Scholar
- Martin Fowler. 1999. Refactoring: Improving the Design of Existing Code (Addison-Wesley Object Technology Series). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA. Google ScholarDigital Library
- Luca Gazzola, Daniela Micucci, and Leonardo Mariani. 2017. Automatic Software Repair: A Survey. IEEE Transactions on Software Engineering Early Access (2017), 36.Google Scholar
- Alex Groce, Daniel Kroening, and Flavio Lerda. 2004. Understanding Counterexamples with explain. In Proceedings of the 16th International Conference on Computer Aided Verification (CAV ’04). Springer, Berlin, Heidelberg, 453–456.Google ScholarCross Ref
- Rahul Gupta, Soham Pal, Aditya Kanade, and Shirish K. Shevade. 2017. DeepFix: Fixing Common C Language Errors by Deep Learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17). AAAI Press, Palo Alto, CA, USA, 1345–1351.Google Scholar
- https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/ 14603/13921Google Scholar
- Masatomo Hashimoto and Akira Mori. 2008. Diff/TS: A Tool for Fine-Grained Structural Change Analysis. In Proceedings of the 15th Working Conference on Reverse Engineering (WCRE ’08). IEEE Computer Society, Washington, DC, USA, 279–288. Google ScholarDigital Library
- Masatomo Hashimoto and Akira Mori. 2012. Enhancing History-Based Concern Mining With Fine-Grained Change Analysis. In Proceedings of the 16th European Conference on Software Maintenance and Reengineering (CSMR ’12). IEEE Computer Society, Washington, DC, USA, 75–84. Google ScholarDigital Library
- 18Google Scholar
- Masatomo Hashimoto, Akira Mori, and Tomonori Izumida. 2015. A Comprehensive and Scalable Method for Analyzing Fine-Grained Source Code Change Patterns. In Proceedings of the 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER ’15). IEEE Computer Society, Washington, DC, USA, 351–360.Google ScholarCross Ref
- Masatomo Hashimoto, Masaaki Terai, Toshiyuki Maeda, and Kazuo Minami. 2015. Extracting Facts from Performance Tuning History of Scientific Applications for Predicting Effective Optimization Patterns. In Proceedings of the 12th IEEE/ACM Working Conference on Mining Software Repositories (MSR ’15). IEEE Computer Society, Washington, DC, USA, 13–23.Google ScholarDigital Library
- Joern Huxhorn. 2016. {LOGBACK-1183} Message formatting regression. Retrieved August 4, 2018 from https://jira.qos.ch/browse/LOGBACK-1183Google Scholar
- Dennis Jeffrey, Min Feng, Neelam Gupta, and Rajiv Gupta. 2009. BugFix: A Learning-Based Tool to Assist Developers in Fixing Bugs. In Proceedings of the 17th IEEE International Conference on Program Comprehension (ICPC ’09). IEEE Computer Society, Washington, DC, USA, 70–79. 2009.5090029Google ScholarCross Ref
- James A. Jones, Mary Jean Harrold, and John Stasko. 2002. Visualization of Test Information to Assist Fault Localization. In Proceedings of the 24th International Conference on Software Engineering (ICSE ’02). ACM, New York, NY, USA, 467–477. Google ScholarDigital Library
- René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA ’14). ACM, New York, NY, USA, 437–440. Google ScholarDigital Library
- 2628055Google Scholar
- Dongsun Kim, Jaechang Nam, Jaewoo Song, and Sunghun Kim. 2013. Automatic Patch Generation Learned from Human-Written Patches. In Proceedings of the 35th International Conference on Software Engineering (ICSE ’13). IEEE Press, Piscataway, NJ, USA, 802–811. Google ScholarDigital Library
- Bogdan Korel and Janusz Laski. 1988. Dynamic program slicing. Inform. Process. Lett. 29, 3 (1988), 155 – 163. Google ScholarDigital Library
- D. J. Kuck, R. H. Kuhn, D. A. Padua, B. Leasure, and M. Wolfe. 1981. Dependence Graphs and Compiler Optimizations. In Proceedings of the 8th ACM SIGPLANSIGACT Symposium on Principles of Programming Languages (POPL ’81). ACM, New York, NY, USA, 207–218. Google ScholarDigital Library
- Xuan-Bach D. Le, Duc-Hiep Chu, David Lo, Claire Le Goues, and Willem Visser. 2017. S3: Syntax- and Semantic-Guided Repair Synthesis via Programming by Examples. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE ’17)). ACM, New York, NY, USA, 593–604. Google ScholarDigital Library
- Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, and Westley Weimer. 2012. GenProg: A Generic Method for Automatic Software Repair. IEEE Transactions on Software Engineering 38, 1 (2012), 54–72. Google ScholarDigital Library
- Fan Long and Martin Rinard. 2016. Automatic Patch Generation by Learning Correct Code. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’16). ACM, New York, NY, USA, 298–312. Google ScholarDigital Library
- Frédéric Magniez and Michel de Rougemont. 2007. Property Testing of Regular Tree Languages. Algorithmica 49, 2 (2007), 127–146. s00453-007-9028-3 Google ScholarDigital Library
- Matias Martinez, Thomas Durieux, Romain Sommerard, Jifeng Xuan, and Martin Monperrus. 2017. Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset. Empirical Software Engineering 22, 4 (2017), 1936–1964. Google ScholarDigital Library
- Sergey Mechtaev, Jooyong Yi, and Abhik Roychoudhury. 2016. Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis. In Proceedings of the 38th International Conference on Software Engineering (ICSE ’16). ACM, New York, ESEC/FSE ’18, November 4–9, 2018, Lake Buena Vista, FL, USA M. Hashimoto et al. NY, USA, 691–701. Google ScholarDigital Library
- Ghassan Misherghi and Zhendong Su. 2006. HDD: Hierarchical Delta Debugging. In Proceedings of the 28th International Conference on Software Engineering (ICSE ’06). ACM, New York, NY, USA, 142–151. Google ScholarDigital Library
- Hoang Duong Thien Nguyen, Dawei Qi, Abhik Roychoudhury, and Satish Chandra. 2013. SemFix: Program Repair via Semantic Analysis. In Proceedings of the 35th International Conference on Software Engineering (ICSE ’13). IEEE Press, Piscataway, NJ, USA, 772–781. Google ScholarDigital Library
- Chris Parnin and Alessandro Orso. 2011. Are Automated Debugging Techniques Actually Helping Programmers?. In Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA ’11). ACM, New York, NY, USA, 199–209. Google ScholarDigital Library
- Terence Parr. 2016. {ANTLR-1543} Type mismatch between left and right labels in left-recursive rule. Retrieved August 4, 2018 from https://github.com/antlr/ antlr4/issues/1543Google Scholar
- Yu Pei, Carlo A. Furia, Martin Nordio, Yi Wei, Bertrand Meyer, and Andreas Zeller. 2014. Automated Fixing of Programs with Contracts. IEEE Transactions on Software Engineering 40, 5 (2014), 427–449. Google ScholarDigital Library
- 2312918Google Scholar
- Yu Pei, Yi Wei, Carlo A. Furia, Martin Nordio, and Bertrand Meyer. 2011. Code-Based Automated Program Fixing. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering (ASE ’11). IEEE Computer Society, Washington, DC, USA, 392–395. Google ScholarDigital Library
- Kyle Prete, Napol Rachatasumrit, Nikita Sudan, and Miryung Kim. 2010. Templatebased Reconstruction of Complex Refactorings. In Proceedings of the 26th IEEE International Conference on Software Maintenance (ICSM ’10). IEEE Computer Society, Washington, DC, USA, 1–10. Google ScholarDigital Library
- Zichao Qi, Fan Long, Sara Achour, and Martin C. Rinard. 2015. An Analysis of Patch Plausibility and Correctness for Generate-and-Validate Patch Generation Systems. In Proceedings of the 2015 International Symposium on Software Testing and Analysis (ISSTA ’15). ACM, New York, NY, USA, 24–36. 1145/2771783.2771791 Google ScholarDigital Library
- Shruti Raghavan, Rosanne Rohana, David Leon, Andy Podgurski, and Vinay Augustine. 2004. Dex: A Semantic-Graph Differencing Tool for Studying Changes in Large Code Bases. In Proceedings of the 20th IEEE International Conference on Software Maintenance (ICSM ’04). IEEE Computer Society, Washington, DC, USA, 188–197. Google ScholarDigital Library
- Manos Renieris and Steven P. Reiss. 2003. Fault Localization with Nearest Neighbor Queries. In Proceedings of the 18th IEEE International Conference on Automated Software Engineering (ASE ’03). IEEE Computer Society, Washington, DC, USA, 30–39. Google ScholarDigital Library
- Reudismam Rolim, Gustavo Soares, Loris D’Antoni, Oleksandr Polozov, Sumit Gulwani, Rohit Gheyi, Ryo Suzuki, and Björn Hartmann. 2017. Learning Syntactic Program Transformations from Examples. In Proceedings of the 39th International Conference on Software Engineering (ICSE ’17). IEEE Press, Piscataway, NJ, USA, 404–415. Google ScholarDigital Library
- Felipe Matos Santana. 2017. {JSOUP-926} Regression: CSS attribute selector. Retrieved August 4, 2018 from https://github.com/jhy/jsoup/issues/926Google Scholar
- Stelios Sidiroglou-Douskos, Eric Lahtinen, Fan Long, and Martin Rinard. 2015. Automatic Error Elimination by Horizontal Code Transfer Across Multiple Applications. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’15). ACM, New York, NY, USA, 43–54. Google ScholarDigital Library
- Shin Hwei Tan and Abhik Roychoudhury. 2015. relifix: Automated Repair of Software Regressions. In 37th IEEE/ACM International Conference on Software Engineering - Volume 1 (ICSE ’15). IEEE Press, Piscataway, NJ, USA, 471–482. Google ScholarDigital Library
- Chris Thunes. 2013. javalang: pure Python Java parser and tools. https: //github.com/c2nes/javalangGoogle Scholar
- toro. 2016. {HSQLDB-1454} Null Pointer in CASEWHEN and CASE... WHEN use case. Retrieved August 4, 2018 from https://sourceforge.net/p/hsqldb/bugs/1454/Google Scholar
- Linus Torvalds. 2005. Git: fast, scalable, distributed revision control system. https://git-scm.com/Google Scholar
- Nikolaos Tsantalis, Matin Mansouri, Laleh M. Eshkevari, Davood Mazinanian, and Danny Dig. 2018. Accurate and Efficient Refactoring Detection in Commit History. In Proceedings of the 40th International Conference on Software Engineering (ICSE ’18). ACM, New York, NY, USA, 483–494. Google ScholarDigital Library
- tvojeho. 2011. {JEDIT-3517} NPE when scrolling to top after multiline delete (soft wrap). Retrieved August 4, 2018 from https://sourceforge.net/p/jedit/bugs/3517/Google Scholar
- Jari Urpalainen. 2008. An Extensible Markup Language (XML) Patch Operations Framework Utilizing XML Path Language (XPath) Selectors. Retrieved August 4, 2018 from https://www.ietf.org/rfc/rfc5261.txtGoogle Scholar
- Westley Weimer, ThanhVu Nguyen, Claire Le Goues, and Stephanie Forrest. 2009. Automatically Finding Patches Using Genetic Programming. In Proceedings of the 31st International Conference on Software Engineering (ICSE ’09). IEEE Computer Society, Washington, DC, USA, 364–374. Google ScholarDigital Library
- 5070536Google Scholar
- Mark Weiser. 1984. Program Slicing. IEEE Transactions on Software Engineering 10, 4 (July 1984), 352–357. Google ScholarDigital Library
- Jifeng Xuan, Matias Martinez, Favio Demarco, Maxime Clement, Sebastian R. Lamelas Marcote, Thomas Durieux, Daniel Le Berre, and Martin Monperrus. 2017. Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs. IEEE Transactions on Software Engineering 43, 1 (2017), 34–55. Google ScholarDigital Library
- Andreas Zeller. 1999. DD.py: Enhanced Delta Debugging class. Retrieved August 4, 2018 from https://www.st.cs.uni-saarland.de/dd/DD.pyGoogle Scholar
- Andreas Zeller. 1999. Yesterday, My Program Worked. Today, It Does Not. Why?. In Proceedings of the 7th European Software Engineering Conference Held Jointly with the 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE ’99). Springer-Verlag, Berlin, Heidelberg, 253–267. Google ScholarDigital Library
- Andreas Zeller. 2002. Isolating Cause-effect Chains from Computer Programs. In Proceedings of the 10th ACM SIGSOFT Symposium on Foundations of Software Engineering (SIGSOFT ’02/FSE-10). ACM, New York, NY, USA, 1–10. org/10.1145/587051.587053 Google ScholarDigital Library
- Andreas Zeller and Ralf Hildebrandt. 2002. Simplifying and Isolating Failure-Inducing Input. IEEE Transactions on Software Engineering 28, 2 (February 2002), 183–200. Google ScholarDigital Library
- K. Zhang and D. Shasha. 1989. Simple fast algorithms for the editing distance between trees and related problems. SIAM J. Comput. 18, 6 (1989), 1245–1262. Google ScholarDigital Library
- Xiangyu Zhang, Neelam Gupta, and Rajiv Gupta. 2006. Pruning Dynamic Slices with Confidence. In Proceedings of the 27th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI ’06). ACM, New York, NY, USA, 169–180. Google ScholarDigital Library
Index Terms
- Automated patch extraction via syntax- and semantics-aware Delta debugging on source code changes
Recommendations
Probabilistic Delta debugging
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringThe delta debugging problem concerns how to reduce an object while preserving a certain property, and widely exists in many applications, such as compiler development, regression fault localization, and software debloating. Given the importance of delta ...
Practical isolation of failure-inducing changes for debugging regression faults
ASE '12: Proceedings of the 27th IEEE/ACM International Conference on Automated Software EngineeringDuring software evolution, new released versions still contain many bugs. One common scenario is that end users encounter regression faults and submit them to bug tracking systems. Different from in-house regression testing, typically only one test ...
Iterative delta debugging
Automated debugging attempts to locate the reason for a failure. Delta debugging minimizes the difference between two inputs, where one input is processed correctly while the other input causes a failure, using a series of test runs to determine the ...
Comments