Skip to main content
SpringerLink
Log in

WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Configuration tuning is essential to optimize the performance of systems (e.g., databases, key-value stores). High performance usually indicates high throughput and low latency. At present, most of the tuning tasks of systems are performed artificially (e.g., by database administrators), but it is hard for them to achieve high performance through tuning in various types of systems and in various environments. In recent years, there have been some studies on tuning traditional database systems, but all these methods have some limitations. In this article, we put forward a tuning system based on attention-based deep reinforcement learning named WATuning, which can adapt to the changes of workload characteristics and optimize the system performance efficiently and effectively. Firstly, we design the core algorithm named ATT-Tune for WATuning to achieve the tuning task of systems. The algorithm uses workload characteristics to generate a weight matrix and acts on the internal metrics of systems, and then ATT-Tune uses the internal metrics with weight values assigned to select the appropriate configuration. Secondly, WATuning can generate multiple instance models according to the change of the workload so that it can complete targeted recommendation services for different types of workloads. Finally, WATuning can also dynamically fine-tune itself according to the constantly changing workload in practical applications so that it can better fit to the actual environment to make recommendations. The experimental results show that the throughput and the latency of WATuning are improved by 52.6% and decreased by 31%, respectively, compared with the throughput and the latency of CDBTune which is an existing optimal tuning method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. O’Neil P, Cheng E, Gawlick D, O’Neil E. The log-structured merge-tree (LSM-tree). Acta Informatica, 1996, 33(4): 351-385. https://doi.org/10.1007/s002360050048.

    Article  MATH  Google Scholar 

  2. Dong S Y, Callaghan M, Galanis L, Borthakur D, Savor T, Stumm M. Optimizing space amplification in RocksDB. In Proc. the 8th Biennial Conference on Innovative Data Systems Research, Jan. 2017.

  3. Chai Y P, Chai Y F, Wang X, Wei H C, Bao N, Liang Y S. LDC: A lower-level driven compaction method to optimize SSD-oriented key-value stores. In Proc. the 35th IEEE International Conference on Data Engineering, April 2019, pp.722-733. https://doi.org/10.1109/ICDE.2019.00070.

  4. Chai Y P, Chai Y F, Wang X, Wei H C, Wang Y Y. Adaptive lower-level driven compaction to optimize LSM-Tree key-value stores. IEEE Transactions on Knowledge Data Engineering. https://doi.org/10.1109/TKDE.2020.3019264.

  5. Zhu Y Q, Liu J X, Guo M Y, Bao Y G, Ma W L, Liu Z Y, Song K P, Yang Y C. BestConfig: Tapping the performance potential of systems via automatic configuration tuning. In Proc. ACM Symposium on Cloud Computing, Sept. 2017, pp.338-350. https://doi.org/10.1145/3127479.3128605.

  6. Van Aken D, Pavlo A, Gordon G J, Zhang B H. Automatic database management system tuning through large-scale machine learning. In Proc. the 2017 ACM International Conference on Management of Data, May 2017, pp.1009-1024. https://doi.org/10.1145/3035918.3064029.

  7. Zhang J, Liu L, Ran M, Li Z K, Liu Y, Zhou K, Li G L, Xiao Z L, Cheng B, Xing J S, Wang Y T, Cheng T H. An end-to-end automatic cloud database tuning system using deep reinforcement learning. In Proc. the 2019 International Conference on Management of Data, June 2019, pp.415-432. https://doi.org/10.1145/3299869.3300085.

  8. Li G L, Zhou X H, Li S F, Gao B. QTune: A query-aware database tuning system with deep reinforcement learning. Proc. the VLDB Endowment, 2019, 12(12): 2118-2130. https://doi.org/10.14778/3352063.3352129.

  9. Lillicrap T P, Hunt J J, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D. Continuous control with deep reinforcement learning. arXiv:1509.02971, 2015. https://arxiv.org/abs/1509.02971, Jun. 2021.

  10. Van Hasselt H. Double Q-learning. In Proc. the 24th Annual Conference on Neural Information Processing Systems, Dec. 2010, pp.2613-2621.

  11. Kingma D, Ba J. Adam: A method for stochastic optimization. In Proc. the 3rd International Conference on Learning Representations, May 2015.

  12. Munos R, Moore A. Variable resolution discretization in optimal control. Machine Learning, 2002, 49(2/3): 291-323. https://doi.org/10.1023/A:1017992615625.

    Article  MATH  Google Scholar 

  13. Mnih V, Kavukcuoglu K, Silver D et al. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529-533. https://doi.org/10.1038/nature14236.

    Article  Google Scholar 

  14. Ban T W. An autonomous transmission scheme using dueling DQN for D2D communication networks. IEEE Transactions on Vehicular Technology, 2020, 69(12): 16348-16352. https://doi.org/10.1109/TVT.2020.3041458.

    Article  Google Scholar 

  15. Chen L, Hu X M, Tang B, Cheng Y. Conditional DQN-based motion planning with fuzzy logic for autonomous driving. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2020.3025671.

  16. Huang H J, Yang Y C, Wang H, Ding Z G, Sari H, Adachi F. Deep reinforcement learning for UAV navigation through massive MIMO technique. IEEE Transactions on Vehicular Technology, 2020, 69(1): 1117-1121. https://doi.org/10.1109/TVT.2019.2952549.

    Article  Google Scholar 

  17. Li J X, Yao L, Xu X, Cheng B, Ren J K. Deep reinforcement learning for pedestrian collision avoidance and human-machine cooperative driving. Information Sciences, 2020, 532: 110-124. https://doi.org/10.1016/j.ins.2020.03.105.

    Article  Google Scholar 

  18. Yoo H, Kim B, Kim J W, Lee J H. Reinforcement learning based optimal control of batch processes using Monte-Carlo deep deterministic policy gradient with phase segmentation. Computers & Chemical Engineering, 2021, 144: Article No. 107133. https://doi.org/10.1016/j.compchemeng.2020.107133.

  19. He X M, Lu H D, Du M, Mao Y C, Wang K. QoE-based task offloading with deep reinforcement learning in edge-enabled Internet of vehicles. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(4): 2252-2261. https://doi.org/10.1109/TITS.2020.3016002.

    Article  Google Scholar 

  20. Li L Y, Xu H, Ma J, Zhou A Z. Joint EH time and transmit power optimization based on DDPG for EH communications. IEEE Communications Letters, 2020, 24(9): 2043-2046. https://doi.org/10.1109/LCOMM.2020.2999914.

    Article  Google Scholar 

  21. Nguyen D Q, Vien N A, Dang V H, Chung T. Asynchronous framework with Reptile+ algorithm to meta learn partially observable Markov decision process. Applied Intelligence, 2020, 50(11): 4050-4062. https://doi.org/10.1007/s10489-020-01748-7.

    Article  Google Scholar 

  22. Gheisarnejad M, Khooban M H. IoT-based DC/DC deep learning power converter control: Real-time implementation. IEEE Transactions on Power Electronics, 2020, 35(12): 13621-13630. https://doi.org/10.1109/TPEL.2020.2993635.

    Article  Google Scholar 

  23. Tang Z T, Shao K, Zhao D B, Zhu Y H. Recent progress of deep reinforcement learning: From AlphaGo to AlphaGoZero. Control Theory & Applications, 2017, 34(12): 1529-1546. https://doi.org/10.7641/CTA.2017.70808. (in Chinese)

  24. Silver D, Schrittwieser J, Simonyan K et al. Mastering the game of Go without human knowledge. Nature, 2017, 550(7676): 354-359. https://doi.org/10.1038/nature24270.

    Article  Google Scholar 

  25. Ye D H, Chen G B, Zhang W et al. Towards playing full MOBA games with deep reinforcement learning. arXiv:2011.12692, 2020. https://arxiv.org/abs/2011.12692, Dec. 2020.

  26. Li G L. Human-in-the-loop data integration. Proceedings of the VLDB Endowment, 2017, 10(12): 2006-2017. https://doi.org/10.14778/3137765.3137833.

  27. Li G L, Zhou X H, Li S H. XuanYuan: An AI-native database. IEEE Data Engineering Bulletin, 2019, 42(2): 70-81.

    Google Scholar 

  28. Basu D, Lin Q, Chen W, Vo H T, Yuan Z, Senellart P, Bressan S. Regularized cost-model oblivious database tuning with reinforcement learning. In Transactions on Large-Scale Data- and Knowledge-Centered Systems XXVIII, Hameurlain A, Küng J, Wagner R, Chen Q (eds.), Springer, 2016, pp.96-132. https://doi.org/10.1007/978-3-662-53455-7_5.

  29. Sun J, Li G L. An end-to-end learning-based cost estimator. Proceedings of the VLDB Endowment, 2019, 13(3): 307-319. https://doi.org/10.14778/3368289.3368296.

  30. Kraska T, Alizadeh M, Beutel A et al. SageDB: A learned database system. In Proc. the 9th Biennial Conference on Innovative Data Systems Research, Jan. 2019.

  31. Duan S Y, Thummala V, Babu S. Tuning database configuration parameters with iTuned. Proceedings of the VLDB Endowment, 2009, 2(1): 1246-1257. https://doi.org/10.14778/1687627.1687767.

  32. Wei Z J, Ding Z H, Hu J L. Self-tuning performance of database systems based on fuzzy rules. In Proc. the 11th International Conference on Fuzzy Systems and Knowledge Discovery, Aug. 2014, pp.194-198. https://doi.org/10.1109/FSKD.2014.6980831.

  33. Zheng C H, Ding Z H, Hu J L. Self-tuning performance of database systems with neural network. In Proc. the 10th International Conference on Natural Computation, Aug. 2014, pp.1-12. https://doi.org/10.1007/978-3-319-09333-8_1.

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Data Engineering and Knowledge Engineering of Ministry of Education, Renmin University of China, Beijing, 100872, China

    Jia-Ke Ge & Yun-Peng Chai

  2. School of Information, Renmin University of China, Beijing, 100872, China

    Jia-Ke Ge, Yan-Feng Chai & Yun-Peng Chai

  3. College of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan, 030027, China

    Yan-Feng Chai

Authors
  1. Jia-Ke Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan-Feng Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yun-Peng Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yun-Peng Chai.

Supplementary Information

ESM 1

(PDF 253 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ge, JK., Chai, YF. & Chai, YP. WATuning: A Workload-Aware Tuning System with Attention-Based Deep Reinforcement Learning. J. Comput. Sci. Technol. 36, 741–761 (2021). https://doi.org/10.1007/s11390-021-1350-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-021-1350-8

Keywords

Search

Navigation