research-article

Better Early Than Never: Performance Test Acceleration by Regression Test Selection

Authors:

David Georg Reichelt,

Stefan KühneAuthors Info & Claims

ICPE '18: Companion of the 2018 ACM/SPEC International Conference on Performance Engineering

Pages 127 - 130

https://doi.org/10.1145/3185768.3186289

Published: 02 April 2018 Publication History

Get Access

Abstract

Currently, performance tests take much time and are therefore not able to provide fast feedback. Fast feedback on performance tests would support finding performance problems. In order to accelerate performance tests we provide a regression test selection method for performance tests. It is based on test selection by (1) code analysis and (2) trace analysis. We show the efficiency of our approach by comparison with the test selection tools EKSTAZI and Infinitest.

References

[1]

Andy Georges, Dries Buytaert, and Lieven Eeckhout. 2007. Statistically Rigorous Java Performance Evaluation. ACM SIGPLAN Notices Vol. 42, 10 (2007), 57--76.

Digital Library

Google Scholar

[2]

Milos Gligoric, Lamyaa Eloussi, and Darko Marinov. 2015. Ekstazi: Lightweight Test Selection. In Proceedings of the 37th International Conference on Software Engineering. IEEE Press, 713--716.

Digital Library

Google Scholar

[3]

Raoufehsadat Hashemian, Niklas Carlsson, Diwakar Krishnamurthy, and Martin Arlitt. 2017. IRIS: Iterative and Intelligent Experiment Selection Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering. ACM, 143--154.

Digital Library

Google Scholar

[4]

Shaikh Mostafa, Xiaoyin Wang, and Tao Xie. 2017. PerfRanker: Prioritization of Performance Regression Tests for Collection-Intensive Software. In Proceedings of the 26th ACM SIGSOFT ISSTA. ACM, 23--34.

Digital Library

Google Scholar

[5]

Steven P Reiss and Manos Renieris. 2001. Encoding Program Executions. In Proceedings of the 23rd ICSE. IEEE Computer Society, 221--230.

Digital Library

Google Scholar

[6]

Juan Pablo Sandoval Alcocer, Alexandre Bergel, and Marco Tulio Valente. 2016. Learning from Source Code History to Identify Performance Failures Proceedings of ACM/SPEC on International Conference on Performance Engineering. ACM, 37--48.

Digital Library

Google Scholar

[7]

Petr Stefan, Vojtech Horky, Lubomir Bulej, and Petr Tuma. 2017. Unit Testing Performance in Java Projects: Are We There Yet? Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering. ACM, 401--412.

Digital Library

Google Scholar

[8]

André Van Hoorn, Jan Waller, and Wilhelm Hasselbring. 2012. Kieker: A Framework for Application Performance Monitoring and Dynamic Software Analysis. In Proceedings of the 3rd ACM/SPEC ICPE. ACM, 247--248.

Digital Library

Google Scholar

[9]

Dennis Westermann, Rouven Krebs, and Jens Happe. 2011. Efficient Experiment Selection in Automated Software Performance Evaluations European Performance Engineering Workshop. Springer, 325--339.

Digital Library

Google Scholar

[10]

S. Yoo and M. Harman. 2012. Regression Testing Minimization, Selection and Prioritization: a Survey. Software Testing, Verification and Reliability Vol. 22, 2 (2012), 67--120.

Crossref

Google Scholar

Cited By

View all

dos Santos Lde Souza ÉEndo ATrubiani CPinciroli RVijaykumar N(2025)Performance regression testing initiativesInformation and Software Technology10.1016/j.infsof.2024.107641179:COnline publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1016/j.infsof.2024.107641
Trubiani CPinciroli RBiaggi AFontana F(2023)Automated Detection of Software Performance Antipatterns in Java-Based ApplicationsIEEE Transactions on Software Engineering10.1109/TSE.2023.323432149:4(2873-2891)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2023.3234321
Schulz HOkanović Dvan Hoorn ATůma PBourcier JJiang ZBezemer CCortellessa V(2021)Context-tailored Workload Model Generation for Continuous Representative Load TestingProceedings of the ACM/SPEC International Conference on Performance Engineering10.1145/3427921.3450240(21-32)Online publication date: 9-Apr-2021
https://dl.acm.org/doi/10.1145/3427921.3450240
Show More Cited By

Index Terms

Better Early Than Never: Performance Test Acceleration by Regression Test Selection
1. Software and its engineering

Recommendations

Empirical Studies of a Safe Regression Test Selection Technique

Regression testing is an expensive testing procedure utilized to validate modified software. Regression test selection techniques attempt to reduce the cost of regression testing by selecting a subset of a program's existing test suite. Safe regression ...
Regression testing minimization, selection and prioritization: a survey

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test ...
Comparing and combining test-suite reduction and regression test selection
ESEC/FSE 2015: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

Regression testing is widely used to check that changes made to software do not break existing functionality, but regression test suites grow, and running them fully can become costly. Researchers have proposed test-suite reduction and regression test ...

Comments

Information & Contributors

Information

Published In

ICPE '18: Companion of the 2018 ACM/SPEC International Conference on Performance Engineering

April 2018

212 pages

ISBN:9781450356299

DOI:10.1145/3185768

General Chairs:
Katinka Wolter
Free University of Berlin, Germany
,
Will Knottenbelt
Imperial College London, UK
,
Program Chairs:
André van Hoorn
University of Stuttgart, Germany
,
Manoj Nambiar
Tata Consultancy Services, India
,
Heiko Koziolek
ABB, Germany

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Hanns Seidel Foundation
German Federal Ministry of Education and Research

Conference

ICPE '18

Sponsor:

ICPE '18: ACM/SPEC International Conference on Performance Engineering

April 9 - 13, 2018

Berlin, Germany

Acceptance Rates

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
155
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

dos Santos Lde Souza ÉEndo ATrubiani CPinciroli RVijaykumar N(2025)Performance regression testing initiativesInformation and Software Technology10.1016/j.infsof.2024.107641179:COnline publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1016/j.infsof.2024.107641
Trubiani CPinciroli RBiaggi AFontana F(2023)Automated Detection of Software Performance Antipatterns in Java-Based ApplicationsIEEE Transactions on Software Engineering10.1109/TSE.2023.323432149:4(2873-2891)Online publication date: 1-Apr-2023
https://doi.org/10.1109/TSE.2023.3234321
Schulz HOkanović Dvan Hoorn ATůma PBourcier JJiang ZBezemer CCortellessa V(2021)Context-tailored Workload Model Generation for Continuous Representative Load TestingProceedings of the ACM/SPEC International Conference on Performance Engineering10.1145/3427921.3450240(21-32)Online publication date: 9-Apr-2021
https://dl.acm.org/doi/10.1145/3427921.3450240
Whiting E(2021)Granular Modeling of User Experience in Load Testing with Automated UI Tests2021 International Conference on Data and Software Engineering (ICoDSE)10.1109/ICoDSE53690.2021.9648440(1-5)Online publication date: 3-Nov-2021
https://doi.org/10.1109/ICoDSE53690.2021.9648440
Reichelt DKühne SHasselbring WZimmermann TLawall JMarinov D(2019)PeASSProceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering10.1109/ASE.2019.00123(1146-1149)Online publication date: 10-Nov-2019
https://dl.acm.org/doi/10.1109/ASE.2019.00123
Huerta-Guevara OAyala-Rivera VMurphy LPortillo-Dominguez A(2019)DYNAMOJM: A JMeter Tool for Performance Testing Using Dynamic Workload AdaptationTesting Software and Systems10.1007/978-3-030-31280-0_14(234-241)Online publication date: 8-Oct-2019
https://doi.org/10.1007/978-3-030-31280-0_14
Huerta-Guevara OAyala-Rivera VMurphy LPortillo-Dominguez A(2019)Towards an Efficient Performance Testing Through Dynamic Workload AdaptationTesting Software and Systems10.1007/978-3-030-31280-0_13(215-233)Online publication date: 8-Oct-2019
https://doi.org/10.1007/978-3-030-31280-0_13

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Empirical Studies of a Safe Regression Test Selection Technique

Regression testing minimization, selection and prioritization: a survey

Comparing and combining test-suite reduction and regression test selection

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Funding Sources

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations