Abstract
Database schema evolution is one of the grand challenges in data management research and practice. In this paper, we survey available schema evolution benchmarks. We argue that existing benchmarks do not reflect the more recent demands on database applications, such as online data migration for high availability applications, agile release strategies, and schema changes that affect more than one data model. We conclude that a new generation of schema evolution benchmarks is called for, which reflect these new demands, especially in the context of schema evolution in NoSQL data stores.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Was originally http://noc.wikimedia.org/cgi-bin/report.py?db=enwiki&sort=real&limit=50000, not available anymore [11/20/2019].
- 2.
References
Alexe, B., Tan, W.C., Velegrakis, Y.: STBenchmark: towards a benchmark for mapping systems. PVLDB 1(1), 230–244 (2008)
Ambler, S.W., Sadalage, P.J.: Refactoring Databases: Evolutionary Database Design. Addison-Wesley Professional, Boston (2006)
Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking cloud serving systems with YCSB. In: Proceedings of the SoCC 2010. ACM (2010)
Curino, C.A., Tanca, L., Moon, H.J., Zaniolo, C.: Schema evolution in wikipedia: toward a web information system benchmark. In: Proceedings of the ICEIS 2008 (2008)
Fry, J.P., Sibley, E.H.: Evolution of data-base management systems. ACM Comput. Surv. 8(1), 7–42 (1976)
Ghazal, A., et al.: BigBench: towards an industry standard benchmark for big data analytics. In: Proceedings of the SIGMOD 2013 (2013)
Gray, J. (ed.): The Benchmark Handbook for Database and Transaction Systems, 2nd edn. Morgan Kaufmann, Burlington (1993)
Han, R., John, L.K., Zhan, J.: Benchmarking big data systems: a review. IEEE Trans. Serv. Comput. 11(3), 580–597 (2018)
Han, R., Lu, X.: On big data benchmarking. CoRR abs/1402.5194 (2014)
Herrmann, K., Voigt, H., Behrend, A., Rausch, J., Lehner, W.: Living in parallel realities: co-existing schema versions with a bidirectional database evolution language. In: Proceedings of the SIGMOD 2017 (2017)
Hillenbrand, A., Levchenko, M., Störl, U., Scherzinger, S., Klettke, M.: MigCast: putting a price tag on data model evolution in NoSQL data stores. In: Proceedings of the SIGMOD 2019 (2019)
Karimov, J., Rabl, T., Markl, V.: PolyBench: the first benchmark for polystores. In: Nambiar, R., Poess, M. (eds.) TPCTC 2018. LNCS, vol. 11135, pp. 24–41. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11404-6_3
Klettke, M., Störl, U., Shenavai, M., Scherzinger, S.: NoSQL schema evolution and big data migration at scale. In: Proceedings of the SCDM 2016 (2016)
Lin, D.Y., Neamtiu, I.: Collateral evolution of applications and databases. In: Proceedings of the IWPSE-Evol 2009 (2009)
Möller, M.L., Klettke, M., Hillenbrand, A., Störl, U.: Query rewriting for continuously evolving NoSQL databases. In: Laender, A.H.F., Pernici, B., Lim, E.-P., de Oliveira, J.P.M. (eds.) ER 2019. LNCS, vol. 11788, pp. 213–221. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33223-5_18
Qiu, D., Li, B., Su, Z.: An empirical analysis of the co-evolution of schema and code in database applications. In: Proceedings of the ESEC/FSE 2013 (2013)
Brueckner, R.: New TPCx-BB Benchmark Compares Big Data Analytics Systems. https://insidehpc.com/2016/06/transaction-processing-performance-council-tpctm-launches-tpcx-bb-a-new-benchmark-for-big-data-analytics-systems/. Accessed 03 Apr 2020
Roddick, J.F.: A survey of schema versioning issues for database systems. Inf. Softw. Technol. 37(7), 383–393 (1995)
Saur, K., Dumitraş, T., Hicks, M.: Evolving NoSQL databases without downtime. In: Proceedings of the ICSME 2016 (2016)
Scherzinger, S., Klettke, M., Störl, U.: Managing schema evolution in NoSQL data stores. In: Proceedings of the DBPL 2013 (2013)
Skoulis, I., Vassiliadis, P., Zarras, A.V.: Growing up with stability. Inf. Syst. 53(C), 363–385 (2015)
Transaction Processing Council: TPC Benchmarks Overview. http://www.tpc.org/information/benchmarks5.asp. Accessed 28 Nov 2019
Transaction Processing Council: TPC Express Big Bench, TPCx-BB Standard Specification, Version 1.3.1. http://www.tpc.org/tpc_documents_current_versions/pdf/tpcx-bb_v1.3.1.pdf. Accessed 29 Nov 2019
Wang, Y., Dong, J., Shah, R., Dillig, I.: Synthesizing database programs for schema refactoring. In: Proceedings of the PLDI 2019 (2019)
Wevers, L., Hofstra, M., Tammens, M., Huisman, M., van Keulen, M.: A benchmark for online non-blocking schema transformations. In: Proceedings of the DATA 2015 (2015)
Wu, S., Neamtiu, I.: Schema evolution analysis for embedded databases. In: Proceedings of the ICDEW 2011 (2011)
Zhang, C., Lu, J., Xu, P., Chen, Y.: UniBench: a benchmark for multi-model database management Systems. In: Nambiar, R., Poess, M. (eds.) TPCTC 2018. LNCS, vol. 11135, pp. 7–23. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11404-6_2
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Möller, M.L., Scherzinger, S., Klettke, M., Störl, U. (2020). Why It Is Time for Yet Another Schema Evolution Benchmark. In: Herbaut, N., La Rosa, M. (eds) Advanced Information Systems Engineering. CAiSE 2020. Lecture Notes in Business Information Processing, vol 386. Springer, Cham. https://doi.org/10.1007/978-3-030-58135-0_10
Download citation
DOI: https://doi.org/10.1007/978-3-030-58135-0_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58134-3
Online ISBN: 978-3-030-58135-0
eBook Packages: Computer ScienceComputer Science (R0)