ConSMutate: SQL Mutants for Guiding Concolic Testing of Database Applications

Abstract

Database applications are built using two different programming language constructs: one that controls the behavior of the application, also referred to as the host language; and the other that allows the application to access/retrieve information from the backend database, also referred to as the query language. The interplay between these two languages makes testing of database applications a challenging process. Independent approaches have been developed to evaluate test case quality for host languages and query languages. Typically, the quality of test cases for the host language (e.g., Java) is evaluated on the basis of the number of lines, statements and blocks covered by the test cases. High quality test cases for host languages can be automatically generated using recently developed concolic testing techniques, which rely on manipulating and guiding the search of test cases based on carefully comparing the concrete and symbolic execution of the program written in the host language. Query language test case quality (e.g., SQL), on the other hand, is evaluated using mutation analysis, which is considered to be a stronger criterion for assessing quality. In this case, several mutants or variants of the original SQL query are generated and the quality is measured using a metric called mutation score. Higher mutation score indicates higher quality for the test cases. In this paper we present a framework, called ConSMutate, which guides concolic testing using mutation analysis for test case generation for database applications. The novelty of the framework is that it ensures that the test cases are of high quality not only in terms of coverage of code written in the host language, but also in terms of mutant detection of the queries written in the query language. We present a prototype implementation of our technique and show its advantages using two non-trivial case studies.