Authors:
Tobias Landsberg
1
;
Christian Dietrich
2
and
Daniel Lohmann
1
Affiliations:
1
Leibniz Universität Hannover, Germany
;
2
Technische Universität Hamburg, Germany
Keyword(s):
Regression Test Selection, Testing, Continuous Integration, Static Analysis.
Abstract:
We describe TASTING, an approach for efficiently selecting and reusing regression-test executions across program changes, branches, and variants in continuous integration settings. Instead of detecting changes between two variants of the software-under-test, TASTING recursively composes hashes of the defining elements with all their dependencies on AST-level at compile time into a semantic fingerprint of the test and its execution environment. This fingerprint is easy to store and remains stable across changes if the test’s run-time behavior is not affected. Thereby, we can reuse test results across the history, multiple branches, and static compile-time variants. We applied TASTING to three open-source projects (Zephyr, OpenSSL, FFmpeg). Over their development history, we can omit between 10 percent (FFmpeg) and 95 percent (Zephyr) of all test executions at a moderate increase in build time. Furthermore, TASTING enables even higher savings across multiple checkouts (e.g., forks, bra
nches, clones) and static software variants. Over the first changes to 131 OpenSSL forks, TASTING avoids 56 percent redundant test executions; for the Zephyr test matrix (64 variants), we reduce the number of test executions by 94 percent.
(More)