Abstract
More and more modern applications are being deployed on cloud-native systems. High concurrent requests that exceed the processing capacity lead to service failures, and the failure of a single microservice result in the entire application unavailable. Currently, most existing methods rely on manual adjustment of microservice’s circuit breaking strategies based on empirical experience. It is very complex to select an appropriate circuit breaking strategy, considering the processing capacity, ect. In this paper, a reinforcement learning-based method is proposed which tries to select optimal circuit breaking strategy by interacting with the system continuously. The proposed method is evaluated on a real Kubernetes and Istio based cluster. Experimental results demonstrate that our approach achieves a successful service rate of 92.6% for requests while 7.4% of requests experience failures. In contrast, the static circuit breaker achieves an average of 48.4% successful service and 51.6% failure rate.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhang, W.: Improving microservice reliability with Istio (2020)
Sharma, R., Singh, A., Sharma, R., Singh, A.: Istio virtualservice. In: Getting Started with Istio Service Mesh: Manage Microservices in Kubernetes, pp. 137–168 (2020)
Saleh Sedghpour, M.R., Klein, C., Tordsson, J.: An empirical study of service mesh traffic management policies for microservices. In: Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering, pp. 17–27 (2022)
Sedghpour, M.R.S., Klein, C., Tordsson, J.: Service mesh circuit breaker: from panic button to performance management tool. In: Proceedings of the 1st Workshop on High Availability and Observability of Cloud Systems, pp. 4–10 (2021)
Fan, J., Wang, Z., Xie, Y., Yang, Z.: A theoretical analysis of deep q-learning. In: Learning for Dynamics and Control, pp. 486–489. PMLR (2020)
Gu, S., Lillicrap, T., Sutskever, I., Levine, S.: Continuous deep Q-learning with model-based acceleration. In: International conference on machine learning, pp. 2829–2838. PMLR (2016)
Mendona, N.C., Aderaldo, C.M., Cámara, J., Garlan, D.: Model-based analysis of microservice resiliency patterns. In: IEEE International Conference on Software Architecture (ICSA 2020) (2020)
Ponomarev, K.Y.: Attribute-based access control in service mesh. In: 2019 Dynamics of Systems, Mechanisms and Machines (Dynamics), pp. 1–4. IEEE (2019)
Montesi, F., Weber, J.: Circuit breakers, discovery, and API gateways in microservices. arXiv preprint arXiv:1609.05830 (2016)
Song, M., Liu, Q., Haihong, E.: A mirco-service tracing system based on Istio and kubernetes. In: 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), pp. 613–616. IEEE (2019)
Meng, F., Jagadeesan, L., Thottan, M.: Model-based reinforcement learning for service mesh fault resiliency in a web application-level. arXiv preprint arXiv:2110.13621 (2021)
Potthoff, L.: Vergleich von circuit-breaker-implementierungen in service-meshes. Abschlussbericht FEP 2022, 49 (2022)
Larsson, L., Tärneberg, W., Klein, C., Kihl, M., Elmroth, E.: Towards soft circuit breaking in service meshes via application-agnostic caching. arXiv preprint arXiv:2104.02463 (2021)
Ganguli, M., Ranganath, S., Ravisundar, S., Layek, A. Ilangovan, D., Verplanke, E.: Challenges and opportunities in performance benchmarking of service mesh for the edge. In: 2021 IEEE International Conference on Edge Computing (EDGE), pp. 78–85. IEEE (2021)
Cao, K., Liu, Y., Meng, G., Sun, Q.: An overview on edge computing research. IEEE Access 8, 85714–85728 (2020)
Mao, Y., You, C., Zhang, J., Huang, K., Letaief, K.B.: A survey on mobile edge computing: the communication perspective. IEEE Commun. Surv. Tutor. 19(4), 2322–2358 (2017)
Sayfan, G.: Mastering Kubernetes. Packt Publishing Ltd. (2017)
Santos, J., Wauters, T., Volckaert, B., De Turck, F.: Towards network-aware resource provisioning in kubernetes for fog computing applications. In: 2019 IEEE Conference on Network Softwarization (NetSoft), pp. 351–359. IEEE (2019)
Harabi, R.: Managing microservices with Istio service mesh (2019)
Calcote, L., Butcher, Z.: Istio: Up and Running: Using a Service Mesh to Connect, Secure, Control, and Observe. O’Reilly Media (2019)
Zhou, J., Li, X., Wang, Q., Qin, X., Miao, W., Tian, J.: Balancing load: an adaptive traffic management scheme for microservices. In: 2022 IEEE 28th International Conference on Parallel and Distributed Systems (ICPADS), pp. 641–648. IEEE (2023)
Kononenko, O., Baysal, O., Holmes, R., Godfrey, M.W.: Mining modern repositories with elastic search. In: Proceedings of the 11th Working Conference on Mining Software Repositories, pp. 328–331 (2014)
Osband, I., Blundell, C., Pritzel, A., Van Roy, B.: Deep exploration via bootstrapped DQN. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Elfwing, S., Uchibe, E., Doya, K.: Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Netw. 107, 3–11 (2018)
van Baaren, E.-J.: Wikibench: a distributed, Wikipedia based web application benchmark. Master’s thesis, VU University Amsterdam (2009)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (Grant No. 61972202, 61973161, 61991404), the Fundamental Research Funds for the Central Universities (No. 30919011235).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Sun, X., Cui, B., Cai, Z. (2024). Deep Q-Learning Based Circuit Breaking Method for Micro-services in Cloud Native Systems. In: Sun, Y., Lu, T., Wang, T., Fan, H., Liu, D., Du, B. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2023. Communications in Computer and Information Science, vol 2012. Springer, Singapore. https://doi.org/10.1007/978-981-99-9637-7_26
Download citation
DOI: https://doi.org/10.1007/978-981-99-9637-7_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9636-0
Online ISBN: 978-981-99-9637-7
eBook Packages: Computer ScienceComputer Science (R0)
