skip to main content
10.1145/1830483.1830654acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Designing better fitness functions for automated program repair

Published: 07 July 2010 Publication History

Abstract

Evolutionary methods have been used to repair programs automatically, with promising results. However, the fitness function used to achieve these results was based on a few simple test cases and is likely too simplistic for larger programs and more complex bugs. We focus here on two aspects of fitness evaluation: efficiency and precision. Efficiency is an issue because many programs have hundreds of test cases, and it is costly to run each test on every individual in the population. Moreover, the precision of fitness functions based on test cases is limited by the fact that a program either passes a test case, or does not, which leads to a fitness function that can take on only a few distinct values. This paper investigates two approaches to enhancing fitness functions for program repair, incorporating (1) test suite selection to improve efficiency and (2) formal specifications to improve precision. We evaluate test suite selection on 10 programs, improving running time for automated repair by 81%. We evaluate program invariants using the Fitness Distance Correlation (FDC) metric, demonstrating significant improvements and smoother evolution of repairs

References

[1]
R. Al-Ekram, A. Adma, and O. Baysal. diffX: an algorithm to detect changes in multi-version XML documents. In Conference of the Centre for Advanced Studies on Collaborative research, pages 1--11. IBM Press, 2005.
[2]
A. Arcuri and X. Yao. A novel co-evolutionary approach to automatic software bug fixing. In IEEE Congress on Evolutionary Computation, 2008.
[3]
BBC News. Microsoft zune affected by 'bug'. In http://news.bbc.co.uk/2/hi/technology/7806683.stm, Dec. 2008.
[4]
R. V. Binder. Testing object-oriented systems: models, patterns, and tools. Addison-Wesley, 1999.
[5]
A. Eiben and J. Smith. Introduction to Evolutionary Computing. Springer, 2003.
[6]
M. D. Ernst, J. H. Perkins, P. J. Guo, S. McCamant, C. Pacheco, M. S. Tschantz, and C. Xiao. The Daikon system for dynamic detection of likely invariants. Science of Computer Programming, 2007.
[7]
S. Forrest, W. Weimer, T. Nguyen, and C. Le Goues. A genetic programming approach to automated software repair. In GECCO, pages 947--954, 2009.
[8]
J. Grefenstette and J. Fitzpatrick. Genetic search with approximate fitness evaluations. In Int. Conf. on Genetic Algorithms and Their Applications, pages 112--120, 1985.
[9]
M. J. Harrold, R. Gupta, and M. L. Soffa. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol., 2(3):270--285, 1993.
[10]
Y. Jin. A comprehensive survey of fitness approximation in evolutionary computation. Soft Computing, 9(1):3--12, 2005.
[11]
T. Jones and S. Forrest. Fitness distance correlation as a measure of problem difficulty for genetic algorithms. In ICGA, pages 184--192, 1995.
[12]
R. Kohavi. A study of cross-validation and bootstrap for accuracy estimation and model selection. IJCAI, 14(2):1137--1145, 1995.
[13]
J. Law and G. Rothermel. Whole program path-based dynamic impact analysis. In ICSE, pages 308--318, 2003.
[14]
Z. Li, M. Harman, and R. M. Hierons. Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng., 33(4):225--237, 2007.
[15]
B. Liblit, M. Naik, A. X. Zheng, A. Aiken, and M. I. Jordan. Scalable statistical bug isolation. In PLDI, pages 12--15, 2005.
[16]
B. P. Miller, L. Fredriksen, and B. So. An empirical study of the reliability of UNIX utilities. Commun. ACM, 33(12):32--44, 1990.
[17]
A. Orso, T. Apiwattanapong, and M. J. Harrold. Leveraging field data for impact analysis and regression testing. SIGSOFT Softw. Eng. Notes, 28(5):128--137, 2003.
[18]
A. Orso, T. Apiwattanapong, J. Law, G. Rothermel, and M. J. Harrold. An empirical comparison of dynamic impact analysis algorithms. In ICSE, pages 491--500, 2004.
[19]
J. H. Perkins, S. Kim, S. Larsen, S. Amarasinghe, J. Bachrach, M. Carbin, C. Pacheco, F. Sherwood, S. Sidiroglou, G. Sullivan, W.-F. Wong, Y. Zibin, M. D. Ernst, and M. Rinard. Automatically patching errors in deployed software. In SOSP, pages 87--102, October 2009.
[20]
X. Ren, F. Shah, F. Tip, B. G. Ryder, and O. Chesley. Chianti: a tool for change impact analysis of java programs. SIGPLAN Not., 39(10):432--448, 2004.
[21]
G. Rothermel and M. J. Harrold. A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol., 6(2):173--210, 1997.
[22]
R. C. Seacord, D. Plakosh, and G. A. Lewis. Modernizing Legacy Systems: Software Technologies, Engineering Process and Business Practices. Addison-Wesley, 2003.
[23]
A. M. Smith and G. M. Kapfhammer. An empirical study of incorporating cost into test suite reduction and prioritization. In ACM symposium on Applied Computing, pages 461--467, 2009.
[24]
K. Walcott, M. Soffa, G. Kapfhammer, and R. Roos. Time-aware test suite prioritization. In ISSTA, 2006.
[25]
S. Wappler and J. Wegener. Evolutionary unit testing of object-oriented software using strongly-typed genetic programming. In Conference on Genetic and Evolutionary Computation, pages 1925--1932, 2006.
[26]
W. Weimer. Patches as better bug reports. In Generative Programming and Component Engineering, 2006.
[27]
W. Weimer, T. Nguyen, C. Le Goues, and S. Forrest. Automatically finding patches using genetic programming. In ICSE, pages 364--367, 2009.
[28]
T.-L. Yu, D. E. Goldberg, and K. Sastry. Optimal sampling and speed-up for genetic algorithms on the sampled onemax problem. In GECCO, pages 1554--1565, 2003.

Cited By

View all
  • (2024)Speeding Up Genetic Improvement via Regression Test SelectionACM Transactions on Software Engineering and Methodology10.1145/368046633:8(1-31)Online publication date: 23-Jul-2024
  • (2024)Causal Relationships and Programming Outcomes: A Transcranial Magnetic Stimulation ExperimentProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639096(1-13)Online publication date: 20-May-2024
  • (2024)Accelerating Patch Validation for Program Repair With Interception-Based Execution SchedulingIEEE Transactions on Software Engineering10.1109/TSE.2024.335996950:3(618-635)Online publication date: 1-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation
July 2010
1520 pages
ISBN:9781450300728
DOI:10.1145/1830483
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. genetic programming
  2. software engineering
  3. software repair

Qualifiers

  • Research-article

Conference

GECCO '10
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)3
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Speeding Up Genetic Improvement via Regression Test SelectionACM Transactions on Software Engineering and Methodology10.1145/368046633:8(1-31)Online publication date: 23-Jul-2024
  • (2024)Causal Relationships and Programming Outcomes: A Transcranial Magnetic Stimulation ExperimentProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639096(1-13)Online publication date: 20-May-2024
  • (2024)Accelerating Patch Validation for Program Repair With Interception-Based Execution SchedulingIEEE Transactions on Software Engineering10.1109/TSE.2024.335996950:3(618-635)Online publication date: 1-Mar-2024
  • (2023)Fitness Function Using Postconditions for Automated Program Repair2023 IEEE 3rd International Conference on Software Engineering and Artificial Intelligence (SEAI)10.1109/SEAI59139.2023.10217541(11-16)Online publication date: 16-Jun-2023
  • (2023)Using the Deep Learning-Based Approaches for Program Debugging and Repair2023 10th International Conference on Dependable Systems and Their Applications (DSA)10.1109/DSA59317.2023.00059(443-454)Online publication date: 10-Aug-2023
  • (2023)The effect of distance metrics in a general purpose synthesizer of imperative programs: A second empirical study using enlarged search spacesSoftware: Practice and Experience10.1002/spe.328654:3(528-540)Online publication date: 15-Nov-2023
  • (2022)Quality of Automated Program Repair on Real-World DefectsIEEE Transactions on Software Engineering10.1109/TSE.2020.299878548:2(637-661)Online publication date: 1-Feb-2022
  • (2022)Restore: Retrospective Fault Localization Enhancing Automated Program RepairIEEE Transactions on Software Engineering10.1109/TSE.2020.298786248:1(309-326)Online publication date: 1-Jan-2022
  • (2021)More Reliable Test Suites for Dynamic APR by using Counterexamples2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE52982.2021.00032(208-219)Online publication date: Oct-2021
  • (2021)Enhancing Genetic Improvement of Software with Regression Test SelectionProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00120(1323-1333)Online publication date: 22-May-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media