Skip to main content
SpringerLink
Log in

Adaptive processing rate based container provisioning for meshed Micro-services in Kubernetes Clouds

  • Regular Paper
  • Published:
CCF Transactions on High Performance Computing Aims and scope Submit manuscript

Abstract

More and more applications are organized in the form of meshed micro-services which can be deployed on the popular container orchestration platform Kubernetes. Designing appropriate container auto-scaling methods for such applications in Kubernetes is beneficial to reducing costs and guaranteeing Quality of Services (QoS). However, most existing resource provisioning methods focus on a service without considering interactions among meshed services. Meanwhile, synchronous calls among services have different impacts on the processing ability of containers as the proportion of different business type’s requests changes which is not considered in existing methods too. Therefore, in this article, an adaptive queuing model and queuing-length aware Jackson queuing network based method is proposed. It adjusts the processing rate of containers according to the ratio of synchronous calls and considers queuing tasks when calculating the impact of bottleneck tiers to others. Experiments are performed on a real Kubernetes cluster, which illustrate that the proposal obtains the lowest percentage of Service Level Agreement (SLA)-violations (decreasing about 6.33%-12.29%) with about 0.9% additional costs compared with existing methods of Kubernetes and other latest methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  • Abdullah, M., Iqbal, W., Bukhari, F., Erradi, A.: Diminishing returns and deep learning for adaptive CPU resource allocation of containers. IEEE Trans. Netw. Serv. Manag. 17(4), 2052–2063 (2020). https://doi.org/10.1109/TNSM.2020.3033025

    Article  Google Scholar 

  • Adam, O., Lee, Y.C., Zomaya, A.Y.: Stochastic resource provisioning for containerized multi-tier web services in clouds. IEEE Trans. Parallel Distrib. Syst. 28(7), 2060–2073 (2017). https://doi.org/10.1109/TPDS.2016.2639009

    Article  Google Scholar 

  • Al-Dhuraibi, Y., Paraiso, F., Djarallah, N., Merle, P.: Elasticity in cloud computing: state of the art and research challenges. IEEE Trans. Serv. Comput. 11(2), 430–447 (2018). https://doi.org/10.1109/TSC.2017.2711009

    Article  Google Scholar 

  • AlibabaCloud.: Container service for kubernetes. https://www.alibabacloud.com/product/kubernetes. Accessed 22 Oct 2021

  • Amazon.: Amazon elastic kubernetes service. https://aws.amazon.com/eks/. Accessed 22 Oct 2021

  • Arlitt, M.F., Williamson, C.L.: Web server workload characterization: the search for invariants. In: Proceedings of the 1996 ACM SIGMETRICS international conference on measurement and modeling of computer systems, association for computing machinery, New York, NY, USA, SIGMETRICS ’96, pp. 126–137, (1996). https://doi.org/10.1145/233013.233034

  • Balaji, M., Rao, G.S.V., Kumar, C.A.: A comparitive study of predictive models for cloud infrastructure management. In: 2014 14th IEEE/ACM international symposium on cluster, cloud and grid computing, pp. 923–926, (2014). https://doi.org/10.1109/CCGrid.2014.32

  • Baresi, L., Guinea, S., Leva, A., Quattrocchi. G.: A discrete-time feedback controller for containerized cloud applications. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, association for computing machinery, New York, NY, USA, FSE 2016, pp. 217–228, (2016). https://doi.org/10.1145/2950290.2950328

  • Barrett, E., Howley, E., Duggan, J.: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Comput. Pract. Exp. 25(12), 1656–1674 (2013). https://doi.org/10.1002/cpe.2864

    Article  Google Scholar 

  • Bi, J., Yuan, H., Tie, M., Tan, W.: Sla-based optimisation of virtualised resource for multi-tier web applications in cloud data centres. Enterp. Inf. Syst. 9(7), 743–767 (2015). https://doi.org/10.1080/17517575.2013.830342

    Article  Google Scholar 

  • Cai, Z., Buyya, R.: Inverse queuing model based feedback control for elastic container provisioning of web systems in Kubernetes. IEEE Trans. Comput. (2021). https://doi.org/10.1109/TC.2021.3049598

    Article  MATH  Google Scholar 

  • Cai, Z., Liu, D., Lu, Y., Buyya, R.: Unequal-interval based loosely coupled control method for auto-scaling heterogeneous cloud resources for web applications. Concurr. Comput. Pract. Exp. 32(23), e5926 (2020). https://doi.org/10.1002/cpe.5926

    Article  Google Scholar 

  • Calheiros, R.N., Ranjan, R., Beloglazov, A., De Rose, C.A.F., Buyya, R.: Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw. Pract. Exp. 41(1), 23–50 (2011). https://doi.org/10.1002/spe.995

    Article  Google Scholar 

  • Chen, H., Wang, Q., Palanisamy, B., Xiong, P.: DCM: dynamic concurrency management for scaling n-tier applications in cloud. In: Lee, K., Liu, L. (eds.) 2017 IEEE 37th international conference on distributed computing systems (ICDCS 2017), pp. 2097–2104, (2017). https://doi.org/10.1109/ICDCS.2017.22

  • Delnat. W., Truyen, E., Rafique, A., Van Landuyt, D., Joosen, W.: K8-scalar: a workbench to compare autoscalers for container-orchestrated database clusters. In: Proceedings of the 13th international conference on software engineering for adaptive and self-managing systems, association for computing machinery, New York, NY, USA, SEAMS ’18, pp. 33–39, (2018). https://doi.org/10.1145/3194133.3194162

  • Huang, G., Wang, S., Zhang, M., Li, Y., Qian, Z., Chen, Y., Zhang, S.: Auto scaling virtual machines for web applications with queueing theory. In: 2016 3rd International conference on systems and informatics (ICSAI), pp. 433–438, (2016). https://doi.org/10.1109/ICSAI.2016.7810994

  • Jiang, J., Lu, J., Zhang, G., Long, G.: Optimal cloud resource auto-scaling for web applications. In: Proceedings of the 13th IEEE/ACM international symposium on cluster, cloud, and grid computing, IEEE Press, CCGRID ’13, pp. 58–65, (2013). https://doi.org/10.1109/CCGrid.2013.73

  • Jmeter, A.: Apache jmeter: workload generator. https://jmeter.apache.org/. Accessed 22 Oct 2021

  • Kho Lin, S., Altaf, U., Jayaputera, G., Li, J., Marques, D., Meggyesy, D., Sarwar, S., Sharma, S., Voorsluys, W., Sinnott, R., Novak, A., Nguyen, V., Pash, K.: Auto-scaling a defence application across the cloud using docker and Kubernetes. In: 2018 IEEE/ACM international conference on utility and cloud computing companion (UCC companion), pp. 327–334, (2018). https://doi.org/10.1109/UCC-Companion.2018.00076

  • Kubernetes.: Horizontal pod autoscaler. https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/. Accessed 22 Oct 2021

  • Lei, Y., Cai, Z., Wu, H., Buyya, R.: Cloud resource provisioning and bottleneck eliminating for meshed web systems. In: 2020 IEEE 13th international conference on cloud computing (CLOUD), pp. 512–516, (2020). https://doi.org/10.1109/CLOUD49709.2020.00076

  • Li, H., Venugopal, S.: Using reinforcement learning for controlling an elastic web application hosting platform. In: Proceedings of the 8th ACM international conference on autonomic computing, association for computing machinery, New York, NY, USA, ICAC ’11, pp. 205–208, (2011). https://doi.org/10.1145/1998582.1998630

  • Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014). https://doi.org/10.1007/s10723-014-9314-7

    Article  Google Scholar 

  • Magableh, B., Almiani, M.: A self healing microservices architecture: a case study in docker swarm cluster. In: Barolli, L., Takizawa, M., Xhafa, F., Enokido, T. (eds.) Advanced Information Networking and Applications, pp. 846–858. Springer, Cham (2020)

    Chapter  Google Scholar 

  • Melikov, A.Z., Rustamov, A.M., Sztrik, J.: Queuing management with feedback in cloud computing centers with large numbers of web servers. In: Vishnevskiy, V.M., Kozyrev, D.V. (eds.) Distributed Computer and Communication Networks, pp. 106–119. Springer, Cham (2018)

    Google Scholar 

  • Pan, W., Mu, D., Wu, H., Yao, L.: Feedback control-based qos guarantees in web application servers. In: 2008 10th IEEE international conference on high performance computing and communications, pp. 328–334, (2008). https://doi.org/10.1109/HPCC.2008.106

  • Piraghaj, S.F., Dastjerdi, A.V., Calheiros, R.N., Buyya, R.: Containercloudsim: an environment for modeling and simulation of containers in cloud data centers. Softw. Pract. Exp. 47(4), 505–521 (2017). https://doi.org/10.1002/spe.2422

    Article  Google Scholar 

  • Qu, C., Calheiros, R.N., Buyya, R.: Auto-scaling web applications in clouds: a taxonomy and survey. ACM Comput. Surv. 51, 4 (2018). https://doi.org/10.1145/3148149

    Article  Google Scholar 

  • Stormacq, S.: Lightsail containers: an easy way to run your containers in the cloud. https://aws.amazon.com/cn/blogs/aws/lightsail-containers-an-easy-way-to-run-your-containers-in-the-cloud/. Accessed 22 Oct 2021

  • Toka, L., Dobreff, G., Fodor, B., Sonkoly, B.: Adaptive ai-based auto-scaling for Kubernetes. In: 2020 20th IEEE/ACM international symposium on cluster, cloud and internet computing (CCGRID), IEEE Computer Society, Los Alamitos, CA, USA, pp. 599–608, (2020). https://doi.org/10.1109/CCGrid49817.2020.00-33

  • TraefikLab.: Traefik : Edge router. (2021). https://doc.traefik.io/traefik/

  • Urdaneta, G., Pierre, G., van Steen, M.: Wikipedia workload analysis for decentralized hosting. Comput. Netw. 53(11), 1830–1845 (2009). https://doi.org/10.1016/j.comnet.2009.02.019

    Article  Google Scholar 

  • Wang, Q., Chen, H., Zhang, S., Hu, L., Palanisamy, B.: Integrating concurrency control in n-tier application scaling management in the cloud. IEEE Trans. Parallel Distrib. Syst. 30(4), 855–869 (2019). https://doi.org/10.1109/TPDS.2018.2871086

    Article  Google Scholar 

  • Xu, M., Buyya, R.: Brownoutcon: a software system based on brownout and containers for energy-efficient cloud computing. J. Syst. Softw. 155, 91–103 (2019). https://doi.org/10.1016/j.jss.2019.05.031

    Article  Google Scholar 

  • Zhang, W., Shi, Y., Liu, L., Zhang, S., Zheng, Y., Cui, L., Yu, H.: CTP: a scheduling strategy to smooth response time fluctuations in multi-tier website system. Microprocess. Microsyst. 47, 198–208 (2016). https://doi.org/10.1016/j.micpro.2016.05.017

    Article  Google Scholar 

  • Zhong, Z., Buyya, R.: A cost-efficient container orchestration strategy in kubernetes-based cloud computing infrastructures with heterogeneous resources. ACM Trans. Internet Technol. (2020). https://doi.org/10.1145/3378447

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant No. 61972202, 61872186, 61973161, 61991404), the Fundamental Research Funds for the Central Universities (No. 30919011235).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China

    Hang Wu, Zhicheng Cai, Yamin Lei & Jian Xu

  2. Cloud Computing and Distributed Systems (CLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia

    Rajkumar Buyya

Authors
  1. Hang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhicheng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yamin Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhicheng Cai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, H., Cai, Z., Lei, Y. et al. Adaptive processing rate based container provisioning for meshed Micro-services in Kubernetes Clouds. CCF Trans. HPC 4, 165–181 (2022). https://doi.org/10.1007/s42514-022-00096-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42514-022-00096-x

Keywords

Search

Navigation