Abstract
Tests that modify (i.e., "pollute") the state shared among tests in a test suite are called \polluter tests". Finding these tests is im- portant because they could result in di erent test outcomes based on the order of the tests in the test suite. Prior work has proposed the PolDet technique for nding polluter tests in runs of JUnit tests on a regular Java Virtual Machine (JVM). Given that Java PathFinder (JPF) provides desirable infrastructure support, such as systematically exploring thread schedules, it is a worthwhile attempt to re-implement techniques such as PolDet in JPF. We present a new implementation of PolDet for nding polluter tests in runs of JUnit tests in JPF. We customize the existing state comparison in JPF to support the so-called \common-root iso- morphism" required by PolDet. We find that our implementation is simple, requiring only -200 lines of code, demonstrating that JPF is a sophisticated infrastructure for rapid exploration of re-search ideas on software testing. We evaluate our implementation on 187 test classes from 13 Java projects and nd 26 polluter tests. Our results show that the runtime overhead of PolDet@JPF com- pared to base JPF is relatively low, on average 1.43x. However, our experiments also show some potential challenges with JPF.
- A. Alshammari, C. Morris, M. Hilton, and J. Bell. FlakeFlagger: Predicting akiness without rerunning tests. In ICSE, 2021.Google Scholar
- A. Gambi, J. Bell, and A. Zeller. Practical test dependency detection. In ICST, 2018.Google ScholarCross Ref
- M. Gruber, S. Lukasczyk, F. Kroi', and G. Fraser. An empirical study of aky tests in Python. In ICST, 2021.Google Scholar
- A. Gyori, A. Shi, F. Hariri, and D. Marinov. Reliable testing: Detecting state-polluting tests to prevent test dependency. In ISSTA, 2015. Google ScholarDigital Library
- M. Harman and P. O'Hearn. From start-ups to scale-ups: Opportunities and open problems for static and dynamic program analysis. In SCAM, 2018.Google ScholarCross Ref
- C. Huo and J. Clause. Improving oracle quality by detecting brittle assertions and unused inputs in tests. In FSE, 2014. Google ScholarDigital Library
- JPF Core. https://github.com/javapathfinder/jpf-core.Google Scholar
- W. Lam, R. Oei, A. Shi, D. Marinov, and T. Xie. iDFlakies: A framework for detecting and partially classifying aky tests. In ICST, 2019.Google ScholarCross Ref
- Q. Luo, F. Hariri, L. Eloussi, and D. Marinov. An empirical analysis of aky tests. In FSE, 2014. Google ScholarDigital Library
- B. Soran, and J. Wuttke. Finding bugs by isolating unit tests. In ESEC/FSE, 2011.Google Scholar
- A. Romano, Z. Song, S. Grandhi, W. Yang, and W. Wang. An empirical analysis of UI-based aky tests. In ICSE, 2021.Google Scholar
- N. Sha ei and F. v. Breugel. Automatic handling of native methods in Java PathFinder. In SPIN, 2014.Google Scholar
- A. Shi, W. Lam, R. Oei, T. Xie, and D. Marinov. iFixFlakies: A framework for automatically xing order-dependent aky tests. In ESEC/FSE, 2019. Google ScholarDigital Library
- N. Tillmann and W. Schulte. Parameterized unit tests. In ESEC/SIGSOFT FSE, 2005. Google ScholarDigital Library
- W. Visser, K. Havelund, G. Brat, and S. Park. Model checking programs. In ASE, 2000. Google ScholarDigital Library
- XStream. https://x-stream.github.io.Google Scholar
- S. Zhang, D. Jalali, J. Wuttke, W. Lam, M. D. Ernst, and D. Notkin. Empirically revisiting the test independence assumption. In ISSTA, 2014. Google ScholarDigital Library
Index Terms
Finding Polluter Tests Using Java PathFinder
Recommendations
State extensions for java pathfinder
ICSE '08: Proceedings of the 30th international conference on Software engineeringJava PathFinder (JPF) is an explicit-state model checker for Java programs. JPF implements a backtrackable Java Virtual Machine (JVM) that provides non-deterministic choices and control over thread scheduling. JPF is itself implemented in Java and runs ...
Verifying android applications using Java PathFinder
Mobile application testing is a specialised and complex field. Due to mobile applications' event driven design and mobile runtime environment, there currently exist only a small number of tools to verify these applications. This paper describes the ...
Abstract pathfinder
We present Abstract Pathfinder, an extension to the Java Pathfinder (JPF) verification tool-set that supports data abstraction to reduce the large data domains of a Java program to small, finite abstract domains, making the program more amenable to ...
Comments