Connor Henderson
ABSTRACT
Modern data systems such as IBM Db2 have hundreds of system configuration parameters, ''knobs", which heavily influence the performance of business queries. Manual configuration, ''tuning," by experts is painstaking and time consuming. We propose a query informed tuning system called BLUTune which uses machine learning (ML)-deep reinforcement learning based on advantage actor critic neural networks-to tune configurations within defined resource constraints. We translate high-dimensional query execution plans (QEPs) into a low-dimensional embedding space (QEP2Vec) for input into the ML models. To scale to complex and large workloads, we bootstrap the training process through transfer learning. We first train our model based on the estimated cost of queries; we then fine-tune it based on actual query execution times. We demonstrate by an experimental study over various synthetic and real-world workloads BLUTune's efficiency and effectiveness.
- Dana Van Aken, Dongsheng Yang, Sebastien Brillard, Ari Fiorino, Bohan Zhang, Christian Billian, and Andrew Pavlo. 2021. An Inquiry into Machine Learning-based Automatic Configuration Tuning Services on Real-World Database Management Systems. PVLDB, Vol. 14, 7 (2021), 1241--1253.Google Scholar
Digital Library
- Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath. 2017. Deep Reinforcement Learning: A Brief Survey. Signal Processing Magazine, Vol. 34, 6 (2017), 26--38.Google ScholarCross Ref
- Peter Belknap, Benoit Dageville, Karl Dias, and Khaled Yagoub. 2009. Self-Tuning for SQL Performance in Oracle Database 11g. In ICDE. 1694--1700.Google Scholar
- Benoit Dageville, Dinesh Das, Karl Dias, Khaled Yagoub, Mohamed Zait, and Mohamed Ziauddin. 2004. Automatic SQL Tuning in Oracle 10g. In VLDB. 1098--1109.Google Scholar
- Guilherme Damasio, Spencer Bryson, Vincent Corvinelli, Parke Godfrey, Piotr Mierzejewski, Jaroslaw Szlichta, and Calisto Zuzarte. 2019a. GALO: Guided Automated Learning for re-Optimization. PVLDB, Vol. 12, 12 (2019), 1778--1781.Google ScholarDigital Library
- Guilherme Damasio, Vincent Corvinelli, Parke Godfrey, Piotr Mierzejewski, Alexandar Mihaylov, Jaroslaw Szlichta, and Calisto Zuzarte. 2019b. Guided automated learning for query workload re-optimization. PVLDB, Vol. 12, 12 (2019), 2010--2021.Google ScholarDigital Library
- Guilherme Damasio, Piotr Mierzejewski, Jaroslaw Szlichta, and Calisto Zuzarte. 2016a. OptImatch: Semantic web system for query problem determination. In ICDE. 1334--1337.Google Scholar
- Guilherme Damasio, Piotr Mierzejewski, Jaroslaw Szlichta, and Calisto Zuzarte. 2016b. Query Performance Problem Determination with Knowledge Base in Semantic Web System OptImatch. In EDBT. 515--526.Google Scholar
- Karl Dias, Mark Ramacher, Uri Shaft, Venkateshwaran Venkataramani, and Graham Wood. 2005. Automatic Performance Diagnosis and Tuning in Oracle. In CIDR. 84--94.Google Scholar
- IBM Db2 Documentation. 2022a. Configuration parameters summary. https://www.ibm.com/docs/en/db2/11.5?topic=parameters-configuration-summary. Accessed: 2022-05-01.Google Scholar
- IBM Db2 Documentation. 2022b. Db2 registry and environment variables. https://www.ibm.com/docs/en/db2/11.5?topic=variables-registry-environment. Accessed: 2021--10-04.Google Scholar
- Songyun Duan, Vamsidhar Thummala, and Shivnath Babu. 2009. Tuning Database Configuration Parameters with iTuned. Proc. VLDB Endow. (2009), 1246--1257.Google ScholarDigital Library
- Yihao Feng, Lihong Li, and Qiang Liu. 2019. A Kernel Loss for Solving the Bellman Equation. In NeurIPS. 15430--15441.Google Scholar
- Yaniv Gur, Dongsheng Yang, Frederik Stalschus, and Berthold Reinwald. 2021. Adaptive Multi-Model Reinforcement Learning for Online Database Tuning. In EDBT. 439--444.Google Scholar
- Eva Kwan, Sam Lightstone, K. Bernhard Schiefer, Adam J. Storm, and Leanne Wu. 2003. Automatic Database Configuration for DB2 Universal Database: Compressing Years of Performance Expertise into Seconds of Execution. In Datenbanksysteme fü r Business, Technologie und Web, BTW-Konferenz. 620--629.Google Scholar
- Quoc V. Le and Tomá s Mikolov. 2014. Distributed Representations of Sentences and Documents. In ICML. 1188--1196.Google Scholar
- Alex X. Lee, Anusha Nagabandi, Pieter Abbeel, and Sergey Levine. 2020. Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Model. In NeurIPS. 2094----2100.Google Scholar
- Viktor Leis, Bernhard Radke, Andrey Gubichev, Alfons Kemper, and Thomas Neumann. 2017. Cardinality Estimation Done Right: Index-Based Join Sampling. In CIDR.Google Scholar
- Guoliang Li, Xuanhe Zhou, Shifu Li, and Bo Gao. 2019. QTune: A Query-Aware Database Tuning System with Deep Reinforcement Learning. PVLDB, Vol. 12, 12 (2019), 2118--2130.Google ScholarDigital Library
- Alexandar Mihaylov, Vincent Corvinelli, Parke Godfrey, Piotr Mierzejewski, Jaroslaw Szlichta, and Calisto Zuzarte. 2021. Scalable Learning to Troubleshoot Query Performance Problems. In CIKM. 4016--4025.Google Scholar
- Meikel Pö ss, Raghunath Othayoth Nambiar, and David Walrath. 2007. Why You Should Run TPC-DS: A Workload Analysis. In VLDB. 1138--1149.Google Scholar
- Patricia Selinger, Morton Astrahan, Donald Chamberlin, Raymond Lorie, and Thomas Price. 1979. Access path selection in a relational database management system. In SIGMOD. 23--34.Google Scholar
- Adam J. Storm, Christian Garcia-Arellano, Sam S. Lightstone, Yixin Diao, and M. Surendra. 2006. Adaptive Self-Tuning Memory in Db2. In VLDB.Google Scholar
- David G. Sullivan, Margo I. Seltzer, and Avi Pfeffer. 2004. Using probabilistic reasoning to automate software tuning. In SIGMETRICS. 404--405.Google Scholar
- Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, and Chunfang Liu. 2018. A Survey on Deep Transfer Learning. In ICANN. 270--279.Google Scholar
- Dana Van Aken, Andrew Pavlo, Geoffrey J. Gordon, and Bohan Zhang. 2017. Automatic Database Management System Tuning Through Large-Scale Machine Learning. In SIGMOD. 1009--1024.Google Scholar
- Hado van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double Q-Learning. In AAAI. 2094----2100.Google Scholar
- Gerhard Weikum, Axel Mö nkeberg, Christof Hasse, and Peter Zabback. 2002. Self-tuning Database Technology and Information Services: from Wishful Thinking to Viable Engineering. In VLDB. 20--31.Google Scholar
- Bohan Zhang, Dana Van Aken, Justin Wang, Tao Dai, Shuli Jiang, Jacky Lao, Siyuan Sheng, Andrew Pavlo, and Geoffrey J. Gordon. 2018. A Demonstration of the OtterTune Automatic Database Management System Tuning Service. PVLDB, Vol. 11, 12 (2018), 1910--1913.Google ScholarDigital Library
- Ji Zhang, Yu Liu, Ke Zhou, Guoliang Li, Zhili Xiao, Bin Cheng, Jiashu Xing, Yangtao Wang, Tianheng Cheng, Li Liu, Minwei Ran, and Zekang Li. 2019. An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning. In SIGMOD. 415--432.Google Scholar
- Yuqing Zhu, Jianxun Liu, Mengying Guo, Yungang Bao, Wenlong Ma, Zhuoyue Liu, Kunpeng Song, and Yingchun Yang. 2017. BestConfig: Tapping the Performance Potential of Systems via Automatic Configuration Tuning. In SoCC. 338--350.Google Scholar
Index Terms
- BLUTune: Query-informed Multi-stage IBM Db2 Tuning via ML
Comments