skip to main content
10.1145/2479440.2479449acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

Mutatis mutandis: evaluating DBMS test adequacy with mutation testing

Published:24 June 2013Publication History

ABSTRACT

Testing consumes significant human and machine resources, especially for large, complex systems such as database servers. While a variety of testing approaches have been proposed to improve the efficiency of the testing process, it is difficult to evaluate these approaches. Mutation testing has been proposed as a way to assess the adequacy of a test suite, assigning a score that can be used to compare testing approaches. While promising, serious obstacles appear to prevent mutation testing with large software systems. Recent advances in mutation testing have scaled to medium-sized programs of around 100,000 lines of code but to our knowledge there are no reported studies working with large systems with millions of lines of code and other features of database systems that complicate testing. In this paper we explore using mutation testing on a database server to evaluate its suitability for comparing test suites or testing approaches.

References

  1. Hardik Bati, Leo Giakoumakis, Steve Herbert, and Aleksandras Surna. 2007. A genetic approach for random testing of database systems. In Proceedings of the 33rd international conference on Very large data bases (VLDB '07). VLDB Endowment 1243--1251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. A. DeMillo, R. J. Lipton, and F. G. Sayward. 1978. Hints on Test Data Selection: Help for the Practicing Programmer. Computer 11, 4 (April 1978), 34-41. DOI=http://dx.doi.org/10.1109/C-M.1978.218136 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. R. J. Lipton, "Fault Diagnosis of Computer Programs," student report, Carnegie Mellon Univ., 1971.Google ScholarGoogle Scholar
  4. Yue Jia and Mark Harman. 2011. An Analysis and Survey of the Development of Mutation Testing. IEEE Trans. Softw. Eng. 37, 5 (September 2011), 649-678. DOI= http://dx.doi.org/10.1109/TSE.2010.62 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. David Schuler, Valentin Dallmeier, and Andreas Zeller. 2009. Efficient mutation testing by checking invariant violations. In Proceedings of the eighteenth international symposium on Software testing and analysis (ISSTA '09). ACM, New York, NY, USA, 69-80. DOI= http://doi.acm.org/10.1145/1572272.1572282 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Roland H. Untch. 1992. Mutation-based software testing using program schemata. In Proceedings of the 30th annual Southeast regional conference (ACM-SE 30). ACM, New York, NY, USA, 285-291. DOI=http://doi.acm.org/10.1145/503720.503749 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Mutatis mutandis: evaluating DBMS test adequacy with mutation testing

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      DBTest '13: Proceedings of the Sixth International Workshop on Testing Database Systems
      June 2013
      63 pages
      ISBN:9781450321518
      DOI:10.1145/2479440

      Copyright © 2013 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 24 June 2013

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      DBTest '13 Paper Acceptance Rate9of15submissions,60%Overall Acceptance Rate31of56submissions,55%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader