Skip to main content
SpringerLink
Log in

A model-based DevOps process for development of mathematical database cost models

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Obviously, the complexity of mathematical database cost models increases with the evolution of the database technology brought by emerging hardware and the new deployment platforms (ex. Cloud). This finding raises questions about the reliability of past Cost Models (CMs). Indeed, redesigning a database CM to evaluate the quality of service (QoS) attributes (i.e. response time, energy, sizing, etc.) is becoming a challenging task. First, because developers directly implement the CM by hard coding inside a DBMS without a prior design. Second, due to a lack of a stepwise development process to support an incremental CM design and continuous testing to diagnose errors that occur at each design stage. Moreover, reusing CMs for other purposes is a major issue that necessitates investigations to allow designers reusing and adapting CMs according to their needs. To take up these challenges, we propose a model-based framework for incremental design and continuous testing of Database CMs Specifically, we are motivated by proposing an approach that aims at shifting CMs design from an adhoc design to a structured and shared design by using a set of design guidelines inspired from software engineering practices. Finally, we propose to use the DevOps reuse practices (Continuous Integration/Continuous Delivery: CI/CD) to store the CM under design in a repository after each upgrade to be reused, improved, calibrated, and refined for other purposes. We evaluate our approach against common CM features, and we carry out a comparison with some analytical models from the literature. Findings show that our framework provides a high CM prediction accuracy, and identify the right design components with a precision ranging from 85% to 100%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. https://www.devops-research.com/research.html

  2. http://www.eclipse.org/cdo

  3. Energy (E) and power (P) can be defined as:\(P=\frac{W}{T}\) and \(E=P\times T\); where P, T, W and E represent respectively, a power, a period of time, the total work performed in that period of time, and the energy.

  4. http://www.tpc.org/tpch/

  5. http://www.tpc.org/tpcds/

References

  • Agrawal, S., Chaudhuri, S., Narasayya, V.: Materialized view and index selection tool for microsoft sql server 2000. ACM SIGMOD Record 30(2), 608 (2001)

    Article  Google Scholar 

  • Asperti, A., Padovani, L., Coen, C.S., Guidi, F., Schena, I.: Mathematical knowledge management in helm. Ann. Math. Artif. Intell. 38(1–3), 27–46 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  • Bausch, D., Petrov, I., Buchmann, A.: Making cost-based query optimization asymmetry-aware. In: Proceedings of the 8th International Workshop on Data Management on New Hardware, pp. 24–32 (2012)

  • Brown, D.P., Chaware, J., Koppuravuri, M.: Index selection in a database system. Google Patents. US Patent 7,499,907 (2009)

  • Chaudhuri, S., Narasayya, V.: Self-tuning database systems: a decade of progress. In: Proceedings of the 33rd International Conference on Very Large Data Bases, pp. 3–14 (2007). VLDB Endowment

  • Chikhaoui, A., Boukhalfa, K., Boukhobza, J.: A cost model for hybrid storage systems in a cloud federations. In: 2018 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1025–1034 (2018). IEEE

  • Dageville, B., Das, D., Dias, K., Yagoub, K., Zait, M., Ziauddin, M.: Automatic sql tuning in oracle 10g. In: Proceedings of the Thirtieth International Conference on Very Large Data Bases-Volume 30, pp. 1098–1109 (2004). VLDB Endowment

  • Djilani, Z., Khouri, S.: Understanding user requirements iceberg: semantic based approach. In: Model and Data Engineering, pp. 297–310. Springer, Cham (2015)

    Chapter  Google Scholar 

  • Ebert, C., Gallardo, G., Hernantes, J., Serrano, N.: Devops. Ieee Softw. 33(3), 94–100 (2016)

    Article  Google Scholar 

  • Guo, R.B., Daudjee, K.: Research challenges in deep reinforcement learning-based join query optimization. In: Proceedings of the Third International Workshop on Exploiting Artificial Intelligence Techniques for Data Management, pp. 1–6 (2020)

  • Humar, I., Ge, X., Xiang, L., Jo, M., Chen, M., Zhang, J.: Rethinking energy efficiency models of cellular networks with embodied energy. IEEE Netw. 25(2), 40–49 (2011)

    Article  Google Scholar 

  • Idreos, S., Zoumpatianos, K., Hentschel, B., Kester, M.S., Guo, D.: The data calculator: Data structure design and cost synthesis from first principles and learned cost models. In: Proceedings of the 2018 International Conference on Management of Data, pp. 535–550 (2018)

  • Lamb, A., Fuller, M., Varadarajan, R., Tran, N., Vandiver, B., Doshi, L., Bear, C.: The vertica analytic database: C-store 7 years later. Proc. VLDB Endow. 5(12), 1790–1801 (2012)

    Article  Google Scholar 

  • Lan, H., Bao, Z., Peng, Y.: A survey on advancing the dbms query optimizer: cardinality estimation, cost model, and plan enumeration. Data Sci. Eng. 6(1), 86–101 (2021)

    Article  Google Scholar 

  • Lang, W., Kandhan, R., Patel, J.M.: Rethinking query processing for energy efficiency: slowing down to win the race. IEEE Data Eng. Bull. 34(1), 12–23 (2011)

    Google Scholar 

  • Leis, V., Radke, B., Gubichev, A., Kemper, A., Neumann, T.: Cardinality estimation done right: Index-based join sampling. In: Cidr (2017)

  • Leis, V., Gubichev, A., Mirchev, A., Boncz, P.A., Kemper, A., Neumann, T.: How good are query optimizers, really? PVLDB 9(3), 204–215 (2015)

    Google Scholar 

  • Lu, J., Chen, Y., Herodotou, H., Babu, S.: Speedup your analytics: automatic parameter tuning for databases and big data systems. Proc. VLDB Endow. 12(12), 1970–1973 (2019)

    Article  Google Scholar 

  • Maier, C., Dash, D., Alagiannis, I., Ailamaki, A., Heinis, T.: Parinda: an interactive physical designer for postgresql. In: Proceedings of the 13th International Conference on Extending Database Technology, pp. 701–704 (2010). ACM

  • Marcus, R., Negi, P., Mao, H., Zhang, C., Alizadeh, M., Kraska, T., Papaemmanouil, O., Tatbul, N.: Neo: A learned query optimizer. arXiv preprint arXiv:1904.03711 (2019)

  • McBrien, P., Rizopoulos, N., Smith, A.C.: SQOWL: type inference in an RDBMS. In: ER, pp. 362–376 (2010)

  • Ouared, A., Chadli, A.: Using mde for teaching database query optimizer. In: ENASE, pp. 529–536 (2021)

  • Ouared, A., Kharroubi, F.Z.: Moving database cost models from darkness to light. In: Smart Applications and Data Analysis: Third International Conference, SADASC 2020, Marrakesh, Morocco, June 25–26, 2020, Proceedings 3, pp. 17–32 (2020). Springer

  • Ouared, A., Ouhammou, Y., Bellatreche, L.: Metricstore repository: on the leveraging of performance metrics in databases. In: Proceedings of the Symposium on Applied Computing, pp. 1820–1825 (2017)

  • Ouared, A., Ouhammou, Y., Bellatreche, L.: Towards a model-based collaborative framework for calibrating database cost models. In: ER Forum/Demos, pp. 44–57 (2017)

  • Ouared, A., Ouhammou, Y., Roukh, A.: A meta-advisor repository for database physical design. In: International Conference on Model and Data Engineering, pp. 72–87 (2016). Springer

  • Ouared, A., Ouhammou, Y.: Capitalizing the database cost models process through a service-based pipeline. Concurr. Comput. Pract. Exp. 35, 6463 (2021)

    Article  Google Scholar 

  • Ouared, A., Ouhammou, Y., Bellatreche, L.: Qosmos: Qos metrics management tool suite. Comput. Lang. Syst. Struct. 54, 236–251 (2018)

    Google Scholar 

  • Ouared, A., Amrani, M., Schobbens, P.-Y.: Comorp: rapid prototyping for mathematical database cost models development. J. Comput. Lang. 73, 101173 (2022)

    Article  Google Scholar 

  • Ouared, A., Chadli, A., Daoud, M.A.: Deepcm: deep neural networks to improve accuracy prediction of database cost models. Concurr. Comput. Pract. Exp. 34(10), 6724 (2022)

    Article  Google Scholar 

  • Siddiqui, T., Jindal, A., Qiao, S., Patel, H., Le, W.: Cost models for big data query processing: Learning, retrofitting, and our findings. In: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, pp. 99–113 (2020)

  • Steinberg, D., Budinsky, F., et al.: EMF: Eclipse Modeling Framework, ser. The Eclipse Series, E. Gamma, L. Nackman, and John Wiegand, Eds. Addison-Wesley Professional (2008)

  • Steinberg, D., Budinsky, F., Merks, E., Paternostro, M.: EMF: Eclipse Modeling Framework. Pearson Education, London (2008)

    Google Scholar 

  • Varadarajan, R., Bharathan, V., Cary, A., Dave, J., Bodagala, S.: Dbdesigner: A customizable physical design tool for vertica analytic database. In: 2014 IEEE 30th International Conference on Data Engineering, pp. 1084–1095 (2014). IEEE

  • Woltmann, L., Hartmann, C., Habich, D., Lehner, W.: Machine learning-based cardinality estimation in dbms on pre-aggregated data. arXiv preprint arXiv:2005.09367 (2020)

  • Wu, W., Chi, Y., Zhu, S., Tatemura, J., Hacigumus, H., Naughton, J.F.: Predicting query execution time: Are optimizer cost models really unusable? In: Data Engineering (ICDE), 2013 IEEE 29th International Conference On, pp. 1081–1092 (2013). IEEE

  • Xu, Z., Tu, Y.-C., Wang, X.: Dynamic energy estimation of query plans in database systems. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems, pp. 83–92 (2013). IEEE

  • Xu, Z., Tu, Y.-C., Wang, X.: Pet: reducing database energy cost via query optimization. Proc. VLDB Endow. 5(12), 1954–1957 (2012)

    Article  Google Scholar 

  • Xu, Z., Tu, Y., Wang, X.: PET: reducing database energy cost via query optimization. PVLDB 5(12), 1954–1957 (2012)

    Google Scholar 

  • Zilio, D.C., Zuzarte, C., Lightstone, S., Ma, W., Lohman, G.M., Cochrane, R.J., Pirahesh, H., Colby, L., Gryz, J., Alton, E., et al. Recommending materialized views and indexes with the ibm db2 design advisor. In: Autonomic Computing, 2004. Proceedings. International Conference On, pp. 180–187 (2004). IEEE

Download references

Author information

Author notes

  1. Abdelhafid Chadli, Abdelkader Ouared authors contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Science, Univeristy of Tiaret, BP 78 zaâroura, 14000, Tiaret, Tiaret, Algeria

    Ahmed Chikhaoui, Abdelhafid Chadli & Abdelkader Ouared

Authors
  1. Ahmed Chikhaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelhafid Chadli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdelkader Ouared
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Investigation, Methodology, and Validation done by Chikhaoui, Chadli, and Ouared. Final revision and English polishing done by Chadli and Ouared. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Ahmed Chikhaoui.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chikhaoui, A., Chadli, A. & Ouared, A. A model-based DevOps process for development of mathematical database cost models. Autom Softw Eng 30, 23 (2023). https://doi.org/10.1007/s10515-023-00390-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-023-00390-0

Keywords

Search

Navigation