Abstract
Software testing enable development teams to maintain the quality of a software system while it evolves. The database manipulation code requires special attention in this context. However, it is often neglected and suffers from software maintenance problems. In this paper, we investigate the current state-of-the-practice in testing database manipulation code. We first analyse the code of 72 projects mined from Libraries.io to get an impression of the test coverage for database code. We confirm that the database is poorly tested: 46% of the projects did not cover with tests half of their database access methods, and 33% of the projects did not cover the database code at all. To understand the difficulties in testing database code, we analysed 532 questions on StackExchange sites and deduced a taxonomy. We found that developers mostly look for insights on general best practices to test database access code. They also have more technical questions related to DB handling, mocking, parallelisation or framework/tool usage. This investigation lays the basis for future research on improving database code testing.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
At the time of writing, it has 2.7M unique packages, 33M repositories, and 235M interdependencies between them.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
References
Repl. pkg. https://github.com/csnagy/caise2021-db-manipulation-testing
Alsharif, A., et al.: What factors make SQL test cases understandable for testers? a human study of automated test data generation techniques. In: ICSME (2019)
Arcuri, A., Fraser, G., Just, R.: Private API access and functional mocking in automated unit test generation. In: Proceedings of ICST (2017)
Beller, M., Gousios, G., Panichella, A., Zaidman, A.: When, how, and why developers (do not) test in their ides. In: Proc. ESEC/FSE (2015)
Castelein, J., Aniche, M., Soltani, M., Panichella, A., van Deursen, A.: Search-based test data generation for SQL queries. In: Proceedings of ICSE (2018)
Chays, D., Dan, S., Frankl, P.G., Vokolos, F.I., Weber, E.J.: A framework for testing database applications. In: Proceedings of ISSTA (2000)
Chen, T.H., Shang, W., Hassan, A.E., Nasser, M., Flora, P.: Detecting problems in the database access code of large scale systems. In: Proceedings of ICSE (2016)
Cleve, A., Brogneaux, A., Hainaut, J.: A conceptual approach to database applications evolution. In: Proceedings of ER (2010)
Delplanque, J., Etien, A., Anquetil, N., Ducasse, S.: Recommendations for evolving relational databases. In: Proceedings of CAiSE (2020)
Deng, Y., Frankl, P., Wang, J.: Testing web database applications. SIGSOFT Softw. Eng. Notes 29(5), 1–10 (2004)
Garg, D., Datta, A.: Test case prioritization due to database changes in web applications. In: Proceedings of ICST (2012)
Gonzalez, D., Santos, J.C.S., Popovich, A., Mirakhorli, M., Nagappan, M.: A large-scale study on the usage of testing patterns that address maintainability attributes: patterns for ease of modification, diagnoses, and comprehension. In: MSR (2017)
González, J.R.C., Romero, J.J.F., Guerrero, M.G., Calderón, F.: Multi-class multi-tag classifier system for stackoverflow questions. In: Proceedings of ROPEC (2015)
Ivanković, M., Petrović, G., Just, R., Fraser, G.: Code coverage at Google. In: Proceedings of ESEC/FSE (2019)
Kapfhammer, G.M., Soffa, M.L.: Database-aware test coverage monitoring. In: Proceedings of the 1st India Software Engineering Conference (2008)
Maule, A., Emmerich, W., Rosenblum, D.: Impact analysis of database schema changes. In: Proceedings of ICSE 2008 (2008)
Meurice, L., Nagy, C., Cleve, A.: Detecting and preventing program inconsistencies under database schema evolution. In: Proceedings of QRS (2016)
Meurice, L., Nagy, C., Cleve, A.: Static analysis of dynamic database usage in java systems. In: Proceedings of CAiSE (2016)
Pasternak, B., Tyszberowicz, S., Yehudai, A.: Genutest: a unit test and mock aspect generation tool. In: Hardware and Software: Verification and Testing (2008)
Petersen, K., Vakkalanka, S., Kuzniarz, L.: Guidelines for conducting systematic mapping studies in software engineering: an update. IST 64, 1–18 (2015)
Qiu, D., Li, B., Su, Z.: An empirical analysis of the co-evolution of schema and code in database applications. In: Proceedings of ESEC/FSE (2013)
Ran, L., et al.: Building test cases and oracles to automate the testing of web database applications. Inf. Softw. Technol. 51(2), 460–477 (2009)
Riaz, M., Mendes, E., Tempero, E.: Towards maintainability prediction for relational database-driven software applications: evidence from software practitioners. In: Proceedings of Advances in Software Engineering (2010)
Rosero, R.H., Gómez, O.S., Rafael, G.D.R.: Regression testing of database applications under an incremental software development setting. IEEE Access 5, 18419–18428 (2017)
Spadini, D., Aniche, M., Bruntink, M., Bacchelli, A.: Mock objects for testing java systems. Empirical Softw. Eng. 24(3), 1461–1498 (2018). https://doi.org/10.1007/s10664-018-9663-0
Stonebraker, M., Deng, D., Brodie, M.L.: Application-database co-evolution: a new design and development paradigm. In: New England Database Day (2017)
Trautsch, F., Grabowski, J.: Are there any unit tests? an empirical study on unit testing in open source python projects. In: Proceedings of ICST (2017)
Tuya, J., Suárez-Cabal, M.J., de la Riva, C.: Full predicate coverage for testing SQL database queries. Softw. Testing, Verification Reliab. 20, 237–288 (2010)
Usman, M., Britto, R., Börstler, J., Mendes, E.: Taxonomies in software engineering: a systematic mapping study and a revised taxonomy development method. Inf. Softw. Technol. 85, 43–59 (2017)
Vasilescu, B., Filkov, V., Serebrenik, A.: Stackoverflow and github: Associations between software development and crowdsourced knowledge. In: Proceedings of ICSC (2013)
Vassiliadis, P., Zarras, A.V.: Survival in schema evolution: Putting the lives of survivor and dead tables in counterpoint. In: Proceedings of CAiSE (2017)
Acknowledgements
This work is supported by (a) the F.R.S.-FNRS and FWO-Vlaanderen via the EOS project 30446992 SECO-ASSIST and (b) Flanders Make vzw.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gobert, M., Nagy, C., Rocha, H., Demeyer, S., Cleve, A. (2021). Challenges and Perils of Testing Database Manipulation Code. In: La Rosa, M., Sadiq, S., Teniente, E. (eds) Advanced Information Systems Engineering. CAiSE 2021. Lecture Notes in Computer Science(), vol 12751. Springer, Cham. https://doi.org/10.1007/978-3-030-79382-1_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-79382-1_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-79381-4
Online ISBN: 978-3-030-79382-1
eBook Packages: Computer ScienceComputer Science (R0)