Skip to main content
SpringerLink
Log in

A framework for testing DBMS features

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Testing a specific feature of a DBMS requires controlling the inputs and outputs of the operators in the query execution plan. However, that is practically difficult to achieve because the inputs/outputs of a query depend on the content of the test database. In this paper, we propose a framework to test DBMS features. The framework includes a database generator called QAGen so that the generated test databases are able to meet the test requirements defined on the test queries. The framework also includes a set of tools to automate test case constructions and test executions. A wide range of DBMS feature testing tasks can be facilitated by the proposed framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Ammann, P., Offutt, J.: Using formal methods to derive test frames in category-partition testing. In: Annual Conference on Computer Assurance, pp. 69–80 (1994)

  2. Binnig, C., Kossmann, D., Lo, E.: Reverse query processing. In: ICDE (2007)

  3. Binnig, C., Kossmann, D., Lo, E., Özsu, M.T.: QAGen: generating query-aware test databases. In: SIGMOD, pp. 341–352 (2007)

  4. Bruno, N., Chaudhuri, S.: Flexible database generators. In VLDB, pp. 1097–1107 (2005)

  5. Bruno, N., Chaudhuri, S., Thomas, D.: Generating Queries with Cardinality Constraints for DBMS Testing. TKDE (2006)

  6. Chaudhuri, S., Narasayya, V.: TPC-D data generation with skew. Accessible at http://ftp.research.microsoft.com/users/viveknar/tpcdskew (1997)

  7. Chordia, S., D ettinger, E., Triou, E.: Different query verification approaches used to test entity sql. In: DBTest, p. 7 (2008)

  8. Clarke, E.M., Grumberg, O., Peled, D.A.: Model checking (2000)

  9. Codd E.F.: A relational model of data for large shared data banks. Commun. ACM 13(6), 377–387 (1970)

    Article  MATH  Google Scholar 

  10. Cohen D.M., Dalal S.R., Fredman M.L., Patton G.C.: The aetg system: an approach to testing based on combinatiorial design. IEEE TSE 23(7), 437–444 (1997)

    Google Scholar 

  11. Cohen, M.B., Gibbons, P.B., Mugridge, W.B., Colbourn, C.J.: Constructing test suites for interaction testing. In: ICSE, pp. 38–48 (2003)

  12. Cook, B., Kroening, D., Sharygina, N.: Cogent: Accurate theorem proving for program verification. In: CAV, pp. 296–300 (2005)

  13. DTM Data Generator. http://www.sqledit.com/dg/

  14. Elhemali, M., Giakoumakis, L.: Unit-testing query transformation rules. In: DBTest, p. 3 (2008)

  15. Elkan, C.: A decision procedure for conjunctive query disjointness. In: PODS (1989)

  16. Ganski, R.A., Wong, H.K.T.: Optimization of nested SQL queries revisited. In: SIGMOD, pp. 23–33 (1987)

  17. Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness (1990)

  18. Graefe G.: Query evaluation techniques for large databases. ACM Comput. Surv. 25(2), 73–170 (1993)

    Article  Google Scholar 

  19. Gray, J., Sundaresan, P., Englert, S., Baclawski, K., Weinberger, P.J.: Quickly generating billion-record synthetic databases. In: SIGMOD, pp. 243–252 (1994)

  20. Grindal M., Offutt J., Andler S.F.: Combination testing strategies: a survey. Softw. Test. Verif. Reliab. 15(3), 167–199 (2005)

    Article  Google Scholar 

  21. Houkjær, K., Torp, K., Wind, R.: Simple and realistic data generation. In: VLDB, pp. 1243–1246 (2006)

  22. Ibarra O.H., Kim C.E.: Fast approximation algorithms for the knapsack and sum of subset problems. J. ACM 22(4), 463–468 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  23. IBM DB2 Test Database Generator. http://www-306.ibm.com/software/data/db2imstools/db2tools/db2tdbg/

  24. Kellerer H., Mansini R., Pferschy U., Speranza M.G.: An efficient fully polynomial approximation scheme for the subset-sum problem. J. Comput. Syst. Sci. 66(2), 349–370 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  25. King J.C.: Symbolic execution and program testing. Commun. ACM 19(7), 385–394 (1976)

    Article  MATH  Google Scholar 

  26. Klug A.: Calculating constraints on relational expression. TODS 5(3), 260–290 (1980)

    Article  MATH  Google Scholar 

  27. Kuijpers B.: Introduction to constraint databases. SIGMOD Rec. 31(3), 35–36 (2002)

    Article  Google Scholar 

  28. Lo, E., Binnig, C., Kossmann, D., Ozsu, M.T., Hon, W.-K.: A Framework for Testing DBMS Features. Technical Report. Hong Kong Polytechnic University (2008)

  29. Mannila, H., Räihä, K.-J.: Test data for relational queries. In: PODS, pp. 217–223 (1986)

  30. Mishra, C., Koudas, N., Zuzarte, C.: Generating targeted queries for database testing. In: SIGMOD, pp. 499–510 (2008)

  31. Ostrand, T.J., Balcer, M.J.: The category-partition method for specifying and generating fuctional tests. Commun. ACM, 31(6) (1988)

  32. Poess, M., Stephens, J.M.: Generating thousand benchmark queries in seconds. In: VLDB, pp. 1045–1053 (2004)

  33. Przydatek B.: A fast approximation algorithm for the subset-sum problem. Int. Trans. Oper. Res. 9(4), 437–459 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  34. Slutz, D.R.: Massive Stochastic Testing of SQL. In: VLDB, pp. 618–622 (1998)

  35. Stephens, J.M., Poess, M.: Mudd: a multi-dimensional data generator. In: WOSP, pp. 104–109 (2004)

  36. Williams, A.W., Probert, R.L.: A measure for component interaction test coverage. In: AICCSA, pp. 304–312 (2001)

  37. Zipf, G.: Human Behaviour and the Principle of Least Effort (1949)

Download references

Author information

Authors and Affiliations

  1. Department of Computing, The Hong Kong Polytechnic University, Hung Hom, Hong Kong

    Eric Lo

  2. ETH Zurich, Zurich, Switzerland

    Carsten Binnig & Donald Kossmann

  3. David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada

    M. Tamer Özsu

  4. Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan

    Wing-Kai Hon

Authors
  1. Eric Lo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carsten Binnig
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Donald Kossmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Tamer Özsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wing-Kai Hon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eric Lo.

Additional information

This work is supported by GRF grant PolyU 5250/09 from Hong Kong RGC and ICRG grants (Project Numbers: 1-ZV5R and A-PC0N) from The Hong Kong Polytechnic University.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lo, E., Binnig, C., Kossmann, D. et al. A framework for testing DBMS features. The VLDB Journal 19, 203–230 (2010). https://doi.org/10.1007/s00778-009-0157-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-009-0157-y

Keywords

Search

Navigation