Skip to main content
SpringerLink
Log in

A bi-metric autoscaling approach for n-tier web applications on kubernetes

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Container-based virtualization techniques are becoming an alternative to traditional virtual machines, due to less overhead and better scaling. As one of the most widely used open-source container orchestration systems, Kubernetes provides a built-in mechanism, that is, horizontal pod autoscaler (HPA), for dynamic resource provisioning. By default, scaling pods only based on CPU utilization, a single performance metric, HPA may create more pods than actually needed. Through extensive measurements of a containerized n-tier application benchmark, RUBBoS, we find that excessive pods consume more CPU and memory and even deteriorate response times of applications, due to interference. Furthermore, a Kubernetes service does not balance incoming requests among old pods and new pods created by HPA, due to stateful HTTP.

In this paper, we propose a bi-metric approach to scaling pods by taking into account both CPU utilization and utilization of a thread pool, which is a kind of important soft resource in Httpd and Tomcat. Our approach collects the utilization of CPU and memory of pods. Meanwhile, it makes use of ELBA, a milli-bottleneck detector, to calculate queue lengths of Httpd and Tomcat pods and then evaluate the utilization of their thread pools. Based on the utilization of both CPU and thread pools, our approach could scale up less replicas of Httpd and Tomcat pods, contributing to a reduction of hardware resource utilization. At the same time, our approach leverages preStop hook along with liveness and readiness probes to relieve load imbalance among old Tomcat pods and new ones. Based on the containerized RUBBoS, our experimental results show that the proposed approach could not only reduce the usage of CPU and memory by as much as 14% and 24% when compared with HPA, but also relieve the load imbalance to reduce average response time of requests by as much as 80%. Our approach also demonstrates that it is better to scale pods by multiple metrics rather than a single one.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Xavier M G, Neves M V, Rose C A F D. A super-peer model for resource discovery services in large-scale grids. Future Generation Computer Systems, 2005, 21(8): 299–306

    Google Scholar 

  2. Burns B, Grant B, Oppenheimer D, Brewer E, Wilkes J. Borg, omega, and kubernetes. Queue, 2016, 14(1): 70–93

    Article  Google Scholar 

  3. Wang Q Y, Malkowski S, Jayasinghe D, Xiong P C, Pu C, Kanemasa Y, Kawaba M, Harada L. The impact of soft resource allocation on n-tier application scalability. In: Proceedings of the 25th IEEE International Symposium on Parallel and Distributed Processing. 2011, 1034–1045.

  4. Lai C, Kimball J, Zhu T, Wang Q, Pu C. MilliScope: a fine-grained monitoring framework for performance debugging of n-tier web services. In: Proceedings of the 37th IEEE International Conference on Distributed Computing Systems. 2017, 92–102

  5. Wang Q Y, Kanemasa Y, Li J, Jayasinghe D, Shimizu T, Matsubara M, Kawaba M, Pu C. Detecting transient bottlenecks in n-tier applications through fine-grained analysis. In: Proceedings of the 33rd IEEE International Conference on Distributed Computing Systems. 2013, 31–40

  6. White B, Lepreau J, Stoller L, Ricci R, Guruprasad S, Newbold M, Hibler M, Barb C, Joglekar A. An integrated experimental environment for distributed systems and Networks. In: Proceedings of the 5th Symposium on Operating Systems Design and Implementation. 2002, 255–270

  7. Wang Q Y, Kanemasa Y, Li J, Lai C A, Cho C A, Nomura Y, Pu C. Lightning in the cloud: a study of transient bottlenecks on n-tier web application performance. In: Proceedings of 2014 Conference on Timely Results in Operating Systems. 2014, 1–15

  8. Zhu C P, Zhao Y L, Bo H, Zeng Q H, Ma Y. Runtime support for type-safe and context-based behavior adaptation. Frontiers of Computer Science, 2014, 8(1): 17–32

    Article  MathSciNet  Google Scholar 

  9. Bernstein D. Containers and cloud: from LXC to docker to Kubernetes. IEEE Cloud Computing, 2014, 1(3): 81–84

    Article  Google Scholar 

  10. Pahl C. Containerization and the PaaS cloud. IEEE Cloud Computing, 2015, 2(3): 24–31

    Article  Google Scholar 

  11. Felter W, Ferreira A, Rajamony R, Rubio J. An updated performance comparison of virtual machines and Linux containers In: Proceedings of 2015 IEEE International Symposium on Performance Analysis of Systems and Software. 2015, 171–172

  12. Zhang Q, Liu L, Pu C, Dou Q, Wu L, Zhou W. A comparative study of containers and virtual machines in big data environment. In: Proceedings of the 11th IEEE International Conference on Cloud Computing. 2018, 178–185

  13. Ruan B, Huang H, Wu S, Jin H. A performance study of containers in cloud environment. In: Proceedings of the 11th IEEE International Conference on Cloud Computing. 2016, 343–356

  14. Abdollahi Vayghan L, Saied M A, Toeroe M, Khendek F. Microservice based architecture: towards high-availability for stateful applications with Kubernetes. In: Proceedings of the 19th IEEE International Conference on Software Quality, Reliability and Security. 2019, 176–185

  15. Chang C, Yang S, Yeh E. A Kubernetes-based monitoring platform for dynamic cloud resource provisioning. In: Proceedings of 2017 IEEE Global Communications Conference. 2017, 1–6

  16. Medel V, Tolon C, Arronategu U, Tolosana-Calasanz R, Banares J A, Rana O F. Client-side scheduling based on application characterization on Kubernetes. In: Proceedings of International Conference on the Economics of Grids, Clouds, Systems, and Services. 2017, 162–176.

  17. Kho Lin S, Altaf U, Jayaputera G, Li J, Marques D, Meggyesy D, Sarwar S, Sharma S, Voorsluys W, Sinnott R, Novak A, Nguyen V, Pash K. Auto-scaling a defence application across the cloud using docker and Kubernetes. In: Proceedings of 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion. 2018, 327–334

  18. Sun Y, Meng L, Song Y K. AutoScale: adaptive QoS-aware container-based cloud applications scheduling framework. KSII Transactions on Internet and Information Systems, 2019, 13(6): 2824–2837

    Google Scholar 

  19. Al-Haidari F, Sqalli M, Salah K. Impact of CPU utilization thresholds and scaling size on autoscaling cloud resources. In: Proceedings of the 5th IEEE International Conference on Cloud Computing Technology and Science. 2014, 256–261

  20. Sarajlic S, Chastang J, Marru S, Jeremy F, Mike L. Scaling JupyterHub using Kubernetes on jetstream cloud: platform as a service for research and educational initiatives in the atmospheric sciences. In: Proceedings of the Practice and Experience on Advanced Research Computing. 2018, 1–4

  21. Versluis L, Neacsu M, Iosup A. A trace-based performance study of autoscaling workloads of workflows in datacenters. In: Proceedings of the 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. 2018, 223–232

  22. Ilyushkin A, Ali-Eldin A, Herbst N, Andre B, Alessandro P, Alexandru I. An experimental performance evaluation of autoscalers for complex workflows. ACM Transactions on Modeling and Performance Evaluation of Computing Systems, 2018, 3(2):1–32

    Article  Google Scholar 

  23. Lorido-Botran T, Miguel-Alonso J, Lozano J. A review of auto-scaling techniques for elastic applications in cloud environments. Journal of Grid Computing, 2014, 12(4): 559–592

    Article  Google Scholar 

  24. Shah J, Dubaria D. Building modern clouds: using docker, Kubernetes google cloud platform. In: Proceedings of the 9th IEEE Annual Computing and Communication Workshop and Conference. 2019, 184–189

  25. Wang Q Y, Kanemasa K, Kawaba M, Pu C. When average is no. average: large response time fluctuations in n-tier systems. In: Proceedings of the 9th International Conference on Autonomic Computing. 2012, 33–42

  26. Pu C, Kimball J, Lai C, Zhu T, Li J, Park J, Wang Q Y. The millibottleneck theory of performance bugs, and its experimental verification. In: Proceedings of the 37th IEEE International Conference on Distributed Computing Systems. 2017, 1919–1926

  27. Zhu T, Li J, Kimball J, Park J, Lai C, Pu C, Wang Q Y. Limitations of load balancing mechanisms for n-tier systems in the presence of millibottlenecks. In: Proceedings of the 37th IEEE International Conference on Distributed Computing Systems. 2017, 1367–1377

Download references

Acknowledgements

We would also like to show our gratitude to Rodrigo Alves Lima, a PhD candidate of Computer Science, Georgia Institute of Technology, for sharing his understanding of ELBA. The research has been supported by a grant from NSFC (Grant No.61702063), Fundamental Science and by a grant from Frontier Technology Research Projects of Chongqing (cstc2017jcyjAX0089) and China Scholarship Council (201708505099).

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Chongqing University of Technology, Chongqing, 401135, China

    Changpeng Zhu

  2. School of Computer Science and Engineering, Chongqing University of Technology, Chongqing, 400054, China

    Changpeng Zhu

  3. School of Journalism and New Media, Xi’an Jiaotong University, Xi’an, 710049, China

    Bo Han

  4. School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, 710049, China

    Yinliang Zhao

Authors
  1. Changpeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinliang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Changpeng Zhu.

Additional information

Changpeng Zhu is a lecturer at the School of AI at Chongqing University of Technology, China. He received a PhD in Computer Science from Xi’an Jiaotong University, and a Bachelor’s Degree and a Master’s Degree in Computer Science from Wuhan University, China. His research interests focus on Java virtual machine, type systems, cloud computing and big data.

Bo Han is a professor at School of Journalism and New Media at Xi’an Jiaotong University, China. He received a Bachelor’s Degree and a PhD in Computer Science from Xi’an Jiaotong University, China. His research interests focus on software engineering, computer network, cloud computing and new media.

Yinliang Zhao is a professor at the School of Computer Science at Xi’an Jiaotong University, China. He received a PhD in Computer Science from Xi’an Jiaotong University, China. His research interests focus on parallel computing and machine learning.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, C., Han, B. & Zhao, Y. A bi-metric autoscaling approach for n-tier web applications on kubernetes. Front. Comput. Sci. 16, 163101 (2022). https://doi.org/10.1007/s11704-021-0118-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-021-0118-1

Keywords

Search

Navigation