skip to main content
research-article

Finding Polluter Tests Using Java PathFinder

Published:21 July 2021Publication History
Skip Abstract Section

Abstract

Tests that modify (i.e., "pollute") the state shared among tests in a test suite are called \polluter tests". Finding these tests is im- portant because they could result in di erent test outcomes based on the order of the tests in the test suite. Prior work has proposed the PolDet technique for nding polluter tests in runs of JUnit tests on a regular Java Virtual Machine (JVM). Given that Java PathFinder (JPF) provides desirable infrastructure support, such as systematically exploring thread schedules, it is a worthwhile attempt to re-implement techniques such as PolDet in JPF. We present a new implementation of PolDet for nding polluter tests in runs of JUnit tests in JPF. We customize the existing state comparison in JPF to support the so-called \common-root iso- morphism" required by PolDet. We find that our implementation is simple, requiring only -200 lines of code, demonstrating that JPF is a sophisticated infrastructure for rapid exploration of re-search ideas on software testing. We evaluate our implementation on 187 test classes from 13 Java projects and nd 26 polluter tests. Our results show that the runtime overhead of PolDet@JPF com- pared to base JPF is relatively low, on average 1.43x. However, our experiments also show some potential challenges with JPF.

References

  1. A. Alshammari, C. Morris, M. Hilton, and J. Bell. FlakeFlagger: Predicting akiness without rerunning tests. In ICSE, 2021.Google ScholarGoogle Scholar
  2. A. Gambi, J. Bell, and A. Zeller. Practical test dependency detection. In ICST, 2018.Google ScholarGoogle ScholarCross RefCross Ref
  3. M. Gruber, S. Lukasczyk, F. Kroi', and G. Fraser. An empirical study of aky tests in Python. In ICST, 2021.Google ScholarGoogle Scholar
  4. A. Gyori, A. Shi, F. Hariri, and D. Marinov. Reliable testing: Detecting state-polluting tests to prevent test dependency. In ISSTA, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Harman and P. O'Hearn. From start-ups to scale-ups: Opportunities and open problems for static and dynamic program analysis. In SCAM, 2018.Google ScholarGoogle ScholarCross RefCross Ref
  6. C. Huo and J. Clause. Improving oracle quality by detecting brittle assertions and unused inputs in tests. In FSE, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. JPF Core. https://github.com/javapathfinder/jpf-core.Google ScholarGoogle Scholar
  8. W. Lam, R. Oei, A. Shi, D. Marinov, and T. Xie. iDFlakies: A framework for detecting and partially classifying aky tests. In ICST, 2019.Google ScholarGoogle ScholarCross RefCross Ref
  9. Q. Luo, F. Hariri, L. Eloussi, and D. Marinov. An empirical analysis of aky tests. In FSE, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. B. Soran, and J. Wuttke. Finding bugs by isolating unit tests. In ESEC/FSE, 2011.Google ScholarGoogle Scholar
  11. A. Romano, Z. Song, S. Grandhi, W. Yang, and W. Wang. An empirical analysis of UI-based aky tests. In ICSE, 2021.Google ScholarGoogle Scholar
  12. N. Sha ei and F. v. Breugel. Automatic handling of native methods in Java PathFinder. In SPIN, 2014.Google ScholarGoogle Scholar
  13. A. Shi, W. Lam, R. Oei, T. Xie, and D. Marinov. iFixFlakies: A framework for automatically xing order-dependent aky tests. In ESEC/FSE, 2019. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. N. Tillmann and W. Schulte. Parameterized unit tests. In ESEC/SIGSOFT FSE, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. W. Visser, K. Havelund, G. Brat, and S. Park. Model checking programs. In ASE, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. XStream. https://x-stream.github.io.Google ScholarGoogle Scholar
  17. S. Zhang, D. Jalali, J. Wuttke, W. Lam, M. D. Ernst, and D. Notkin. Empirically revisiting the test independence assumption. In ISSTA, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Finding Polluter Tests Using Java PathFinder
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGSOFT Software Engineering Notes
      ACM SIGSOFT Software Engineering Notes  Volume 46, Issue 3
      July 2021
      40 pages
      ISSN:0163-5948
      DOI:10.1145/3468744
      Issue’s Table of Contents

      Copyright © 2021 Copyright is held by the owner/author(s)

      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 21 July 2021

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader