poster

Search-based mutation testing to improve performance tests

Authors:

Ana B. Sánchez,

Pedro Delgado-Pérez,

Inmaculada Medina-Bulo,

Sergio SeguraAuthors Info & Claims

GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

Pages 316 - 317

https://doi.org/10.1145/3205651.3205670

Published: 06 July 2018 Publication History

Get Access

Abstract

Performance bugs are common and can cause a significant deterioration in the behaviour of a program, leading to costly issues. To detect them and reduce their impact, performance tests are typically applied. However, there is a lack of mechanisms to evaluate the quality of performance tests, causing many of these bugs remain unrevealed. Mutation testing, a fault-based technique to assess and improve test suites, has been successfully studied with functional tests. In this paper, we propose the use of mutation testing together with a search-based strategy (evolutionary algorithm) to find mutants that simulate performance issues. This novel approach contributes to enhance the confidence on performance tests while reducing the cost of mutation testing.

References

[1]

P. Chevalley. 2001. Applying mutation analysis for object-oriented programs using a reflective approach. In Proceedings Eighth Asia-Pacific Sofi ware Engineering Conference. 267--270.

Digital Library

Google Scholar

[2]

P. Delgado-Pérez, I. Medina-Bulo, and M. Núñez. 2017. Using Evolutionary Mutation Testing to improve the quality of test suites. In Proceedings of the IEEE Congress on Evolutionary Computation, CEC'17. 596--603.

Google Scholar

[3]

R. A. DeMillo, R. J. Lipton, and F. G. Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11, 4 (April 1978), 34--41.

Digital Library

Google Scholar

[4]

G. Jin, L. Song, X. Shi, J. Scherpelz, and S. Lu. 2012. Understanding and Detecting Real-world Performance Bugs. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '12). ACM, New York, NY, USA, 77--88.

Digital Library

Google Scholar

[5]

I. Molyneaux. 2009. The Art of Application Performance Testing: Help for Programmers and Quality Assurance. O'Reilly Media.

Digital Library

Google Scholar

[6]

A. Nistor, T. Jiang, and L. Tan. 2013. Discovering, reporting, and fixing performance bugs. In Working Conference on Mining Software Repositories. 237--246.

Digital Library

Google Scholar

[7]

S. Segura, J. Troya, A. Durán, and A. Ruiz-Cortés. 2017. Performance Metamorphic Testing: Motivation and Challenges. In International Conference on Software Engineering: New Ideas and Emerging Results Track. 7--10.

Digital Library

Google Scholar

Cited By

View all

Koroglu YWotawa F(2023) Bottleneck Analysis via Grammar-based Performance Fuzzing * 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW58534.2023.00041(180-185)Online publication date: Apr-2023
https://doi.org/10.1109/ICSTW58534.2023.00041
Delgado‐Pérez PSánchez ASegura SMedina‐Bulo I(2020)Performance mutation testingSoftware Testing, Verification and Reliability10.1002/stvr.172831:5Online publication date: 29-Jan-2020
https://doi.org/10.1002/stvr.1728
Chen YWinter SSuri N(2019)Inferring Performance Bug Patterns from Developer Commits2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2019.00017(70-81)Online publication date: Oct-2019
https://doi.org/10.1109/ISSRE.2019.00017

Index Terms

Search-based mutation testing to improve performance tests
1. Software and its engineering
  1. Software creation and management
    1. Search-based software engineering
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Predictive mutation testing
ISSTA 2016: Proceedings of the 25th International Symposium on Software Testing and Analysis

Mutation testing is a powerful methodology for evaluating test suite quality. In mutation testing, a large number of mutants are generated and executed against the test suite to check the ratio of killed mutants. Therefore, mutation testing is widely ...
Search-Based Higher Order Mutation Testing: A Mapping Study
SAST '18: Proceedings of the III Brazilian Symposium on Systematic and Automated Software Testing

Higher Order Mutation Testing (HOMT) uses mutants derived by applying mutation operators more than once in the program under test. This kind of test increases efficacy, but can be more expensive and lead to some challenges such as the large space of ...
Mitigating the effects of flaky tests on mutation testing
ISSTA 2019: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

Mutation testing is widely used in research as a metric for evaluating the quality of test suites. Mutation testing runs the test suite on generated mutants (variants of the code under test), where a test suite kills a mutant if any of the tests fail ...

Comments

Information & Contributors

Information

Published In

GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion

July 2018

1968 pages

ISBN:9781450357647

DOI:10.1145/3205651

Editor:
Hernan Aguirre
Shinshu University
,
General Chair:
Keiki Takadama
The University of Electro-Communications

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 July 2018

Check for updates

Author Tags

Qualifiers

Poster

Conference

GECCO '18

Sponsor:

SIGEVO

GECCO '18: Genetic and Evolutionary Computation Conference

July 15 - 19, 2018

Kyoto, Japan

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
181
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Koroglu YWotawa F(2023) Bottleneck Analysis via Grammar-based Performance Fuzzing * 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW58534.2023.00041(180-185)Online publication date: Apr-2023
https://doi.org/10.1109/ICSTW58534.2023.00041
Delgado‐Pérez PSánchez ASegura SMedina‐Bulo I(2020)Performance mutation testingSoftware Testing, Verification and Reliability10.1002/stvr.172831:5Online publication date: 29-Jan-2020
https://doi.org/10.1002/stvr.1728
Chen YWinter SSuri N(2019)Inferring Performance Bug Patterns from Developer Commits2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE.2019.00017(70-81)Online publication date: Oct-2019
https://doi.org/10.1109/ISSRE.2019.00017

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Predictive mutation testing

Search-Based Higher Order Mutation Testing: A Mapping Study

Mitigating the effects of flaky tests on mutation testing

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations