short-paper

Enabling Containerized, Parametric and Distributed Database Deployment and Benchmarking as a Service

Authors:

George Kousiouris,

Dimosthenis KyriazisAuthors Info & Claims

ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering

Pages 77 - 80

https://doi.org/10.1145/3447545.3451188

Published: 19 April 2021 Publication History

Abstract

Containerized environments introduce a set of performance challenges that require extensive measurements and benchmarking to identify and model application behavior regarding a variety of parameters. Databases present extra challenges given their extensive need for synchronization and orchestration of a benchmark run, especially in microservice-oriented technologies (such as container platforms) and dynamic business models such as DBaaS. In this work we describe the adaptation of our open source, baseline load injection as a service tool, Flexibench, in order to enable the automated, parametric launching and measurement of containerized and distributed databases as a service. Adaptation and synchronization needs are described for ensuring test sequence and applied through a case study on MySQL. Therefore a performance engineer can directly test selected configuration and performance of a database in a given target workload with simple REST invocations. Experimentation starts from adapting the official MySQL docker images as well as OLTP Bench Client ones and investigates scenarios such as parameter sweep experiments and co-allocation scenarios where multiple DB instances are sharing physical nodes, as expected in the DBaaS paradigm.

References

[1]

M. Grambow, J. Hasenburg, T. Pfandzelter, and D. Bermbach. 2019. Is it safe to dockerize my database benchmark? In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (SAC '19). Association for Computing Machinery, New York, NY, USA,341--344.

Digital Library

[2]

Silva, M., Hines, M.R., Gallo, D., Liu, Q., Ryu, K.D. and Da Silva, D., 2013, March. Cloudbench: Experiment automation for cloud environments. In 2013 IEEE International Conference on Cloud Engineering (IC2E) (pp. 302--311). IEEE.

Digital Library

[3]

Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R. and Sears, R., 2010, June. Benchmarking cloud serving systems with YCSB. In Proceedings of the 1st ACM symposium on Cloud computing (pp. 143--154). ACM.

[4]

Kyriazis, D., Doulkeridis, C., Gouvas, P., Jimenez-Peris, R., Ferrer, A.J., Kallipolitis, L., Kranas, P., Kousiouris, G., Macdonald, C., McCreadie, R. and Moatti, Y., 2018, July. BigDataStack: A holistic data-driven stack for big data applications and operations. In 2018 IEEE International Congress on Big Data (BigData Congress) (pp. 237--241)

[5]

Seybold, D., Volpert, S., Wesner, S., Bauer, A., Herbst, N.R. and Domaschka, J., 2020, January. Kaa: evaluating elasticity of cloud-hosted DBMS. In Proceedings. The 11th IEEE International Conference on Cloud Computing Technology and Science (CloudCom 2019) (Vol. 2019, pp. 54--61).

[6]

Osman, A., Hanisch, S. and Strufe, T., 2019, June. SeCoNetBench: A modular framework for Secure Container Networking Benchmarks. In 2019 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) (pp. 21--28). IEEE.

[7]

Wang, Y., Evans, R.T. and Huang, L., 2019. Performant container support for HPC applications. In Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning) (pp. 1--6).

[8]

Kozhirbayev, Z. and Sinnott, R.O., 2017. A performance comparison of container-based technologies for the cloud. Future Generation Computer Systems, 68, pp.175--182.

[9]

Bachiega, N.G., Souza, P.S., Bruschi, S.M. and de Souza, S.D.R., 2018, April. Container-based performance evaluation: A survey and challenges. In 2018 IEEE International Conference on Cloud Engineering (IC2E) (pp. 398--403). IEEE.

[10]

Gan, Y., Zhang, Y., Cheng, D., Shetty, A., Rathi, P., Katarki, N., Bruno, A., Hu, J., Ritchken, B., Jackson, B. and Hu, K., 2019, April. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (pp. 3--18).

[11]

Sriraman, A. and Wenisch, T.F., 2018, September. ? Suite: A Benchmark Suite for Microservices. In 2018 IEEE International Symposium on Workload Characterization (IISWC) (pp. 1--12). IEEE.

[12]

v. Kistowski, J., Eismann, S., Grohmann, J., Schmitt, N., Bauer, A. and Kounev, S., 2019, March. TeaStore-A Micro-Service Reference Application for Performance Engineers. In Companion of the 2019 ACM/SPEC International Conference on Performance Engineering (pp. 47--48).

[13]

Kousiouris, G., Kyriazis, D., Gogouvitis, S., Menychtas, A., Konstanteli, K. and Varvarigou, T., 2011, June. Translation of application-level terms to resource-level attributes across the Cloud stack layers. In 2011 IEEE Symposium on Computers and Communications (ISCC) (pp. 153--160).

[14]

BigDataStack Project Deliverable 5.1, Available at: https://bigdatastack.eu/deliverables/d51-dimensioning-modelling-and-interaction-services-bigdatastack

[15]

Difallah, D.E., Pavlo, A., Curino, C. and Cudre-Mauroux, P., 2013. Oltp-bench: An extensible testbed for benchmarking relational databases. Proceedings of the VLDB Endowment, 7(4), pp.277--288.

Digital Library

[16]

Seybold, D., Keppler, M., Gründler, D. and Domaschka, J., 2019, April. Mowgli: Finding your way in the DBMS jungle. In Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (pp. 321--332).

Cited By

Jha DLi YWen ZMorgan GJayaraman PKoutny MRana ORanjan R(2024)GeoDeploy: Geo-Distributed Application Deployment Using BenchmarkingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.347053235:12(2361-2374)Online publication date: Dec-2024
https://doi.org/10.1109/TPDS.2024.3470532
Sergeev ARezedinova EKhakhina A(2022)Docker Container Performance Comparison on Windows and Linux Operating Systems2022 International Conference on Communications, Information, Electronic and Energy Systems (CIEES)10.1109/CIEES55704.2022.9990683(1-4)Online publication date: 24-Nov-2022
https://doi.org/10.1109/CIEES55704.2022.9990683
Seybold DDomaschka JZhang KGherbi AVenkatasubramanian NVeiga L(2021)Benchmarking-as-a-service for cloud-hosted DBMSProceedings of the 22nd International Middleware Conference: Demos and Posters10.1145/3491086.3492473(12-13)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1145/3491086.3492473

Index Terms

Enabling Containerized, Parametric and Distributed Database Deployment and Benchmarking as a Service
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
2. Information systems
  1. Data management systems

Recommendations

Benchmarking Database Cloud Services
Performance Evaluation and Benchmarking for the Era of Cloud(s)
Abstract
Meanwhile, many database cloud services are available. The well-known providers are AWS (Amazon Web Service), Google Cloud, and Azure (Microsoft). Oracle and IBM offer cloud services for their in-house database products. In the past, the TPC ...
Benchmarking simple database operations
SIGMOD '87: Proceedings of the 1987 ACM SIGMOD international conference on Management of data

There are two widely-known benchmarks for database management systems the TP1 benchmarks (Anon et al [1985]), designed to measure transaction throughout, and the Wisconsin benchmarks (Bitton, Dewitt, & Turbyfil [1984]), designed to measure the ...
Supporting local autonomy in a distributed object-oriented database

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering

April 2021

198 pages

ISBN:9781450383318

DOI:10.1145/3447545

General Chairs:
Johann Bourcier
University of Rennes 1, France
,
Zhen Ming (Jack) Jiang
York University, Canada
,
Program Chairs:
Cor-Paul Bezemer
University of Alberta, Canada
,
Vittorio Cortellessa
University of L'Aquila, Italy

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

European Commission

Conference

ICPE '21

Sponsor:

ICPE '21: ACM/SPEC International Conference on Performance Engineering

April 19 - 23, 2021

Virtual Event, France

Acceptance Rates

Overall Acceptance Rate 252 of 851 submissions, 30%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
111
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jha DLi YWen ZMorgan GJayaraman PKoutny MRana ORanjan R(2024)GeoDeploy: Geo-Distributed Application Deployment Using BenchmarkingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.347053235:12(2361-2374)Online publication date: Dec-2024
https://doi.org/10.1109/TPDS.2024.3470532
Sergeev ARezedinova EKhakhina A(2022)Docker Container Performance Comparison on Windows and Linux Operating Systems2022 International Conference on Communications, Information, Electronic and Energy Systems (CIEES)10.1109/CIEES55704.2022.9990683(1-4)Online publication date: 24-Nov-2022
https://doi.org/10.1109/CIEES55704.2022.9990683
Seybold DDomaschka JZhang KGherbi AVenkatasubramanian NVeiga L(2021)Benchmarking-as-a-service for cloud-hosted DBMSProceedings of the 22nd International Middleware Conference: Demos and Posters10.1145/3491086.3492473(12-13)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1145/3491086.3492473
Kousiouris G(2021)A self-adaptive batch request aggregation pattern for improving resource management, response time and costs in microservice and serverless environments2021 IEEE International Performance, Computing, and Communications Conference (IPCCC)10.1109/IPCCC51483.2021.9679422(1-10)Online publication date: 29-Oct-2021
https://doi.org/10.1109/IPCCC51483.2021.9679422

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten