skip to main content
10.1145/3412841.3441887acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

Reducing cold starts during elastic scaling of containers in kubernetes

Published: 22 April 2021 Publication History

Abstract

Automatic scaling of containers is an important feature to handle fluctuating workloads. However, the delay caused by the time to bootstrap a container has an impact on applications with deadline-based Service-Level Objectives (SLOs). This delay is called the cold start problem. Many techniques have already been proposed to tackle this problem but not all techniques have been integrated and evaluated in Kubernetes, the de-facto standard in container orchestration. This paper combines and evaluates three techniques in the context of Kubernetes: (i) pre-creation of network containers, (ii) using container images that enable sharing of linked libraries in memory and (iii) extending the declarative configuration management approach of Kubernetes with imperative configuration for creating multiple application containers in parallel. A prototype of the approach is implemented and tested on a Java-based Spring Boot application where the cold start problem occurs due to various library dependencies. Our findings illustrate that the use of containers that allow for library sharing already has a large, positive impact when starting up a single container. The pre-creation of network containers in combination with imperative configuration enables the application to meet deadline-driven SLOs without a non-deterministic delay that appears in Kubernetes when multiple containers are created in parallel. We conclude that the use of container images that allow for library sharing is a must for all applications that require fast container start-ups in Kubernetes. Pre-creation of network containers when combined with imperative configuration also has a positive impact on SLO compliance during elastic scaling of containers.

References

[1]
Ioana Baldini, Paul Castro, Kerry Chang, Perry Cheng, Stephen Fink, Vatche Ishakian, Nick Mitchell, Vinod Muthusamy, Rodric Rabbah, Aleksander Slominski, and Philippe Suter. 2017. Serverless Computing: Current Trends and Open Problems. Springer Singapore, 1--20.
[2]
David Bernstein. 2014. Containers and Cloud: From LXC to Docker to Kubernetes. IEEE Cloud Computing 1, 3 (2014), 81--84.
[3]
U. Breitenb ucher, T. Binz, K. Képes, O. Kopp, F. Leymann, and J. Wettinger. 2014. Combining Declarative and Imperative Cloud Application Provisioning Based on TOSCA. In 2014 IEEE International Conference on Cloud Engineering. 87--96.
[4]
Eric A Brewer. 2015. Kubernetes and the path to cloud native. In Proceedings of the sixth ACM symposium on cloud computing. 167--167.
[5]
Renato Byrro. 2019. Can We Solve Serverless Cold Starts? https://dashbird.io/blog/can-we-solve-serverless-cold-starts/
[6]
James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, and Jonathan Appavoo. 2019. SEUSS: Rapid serverless deployment using environment snapshots. arXiv preprint arXiv:1910.01558 (2019).
[7]
James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, and Jonathan Appavoo. 2019. SEUSS: Rapid serverless deployment using environment snapshots. arXiv preprint arXiv:1910.01558 (2019).
[8]
Lorenzo Civolani, Guillaume Pierre, and Paolo Bellavista. 2019. FogDocker: Start container now, fetch image later. In Proceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing. 51--59.
[9]
José Bravo Ferreira, Marco Cello, and Jesús Omana Iglesias. 2017. More sharing, more benefits? A study of library sharing in container-based infrastructures. In European Conference on Parallel Processing. Springer, 358--371.
[10]
Silvery Fu, Radhika Mittal, Lei Zhang, and Sylvia Ratnasamy. 2020. Fast and Efficient Container Startup at the Edge via Dependency Scheduling. In 3rd {USENIX} Workshop on Hot Topics in Edge Computing (HotEdge 20).
[11]
Tyler Harter, Brandon Salmon, Rose Liu, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2016. Slacker: Fast distribution with lazy docker containers. In 14th {USENIX} Conference on File and Storage Technologies ({FAST} 16). 181--195.
[12]
Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Yadwadkar, et al. 2019. Cloud programming simplified: A berkeley view on serverless computing. arXiv preprint arXiv:1902.03383 (2019).
[13]
Kubernetes. 2020. Kubernetes Documentation. https://kubernetes.io/docs/concepts/workloads/pods/
[14]
Ian Lewis. 2017. The Almighty Pause Container. https://www.ianlewis.org/en/almighty-pause-container
[15]
Ping-Min Lin and Alex Glikson. 2019. Mitigating Cold Starts in Serverless Platforms: A Pool-Based Approach. arXiv preprint arXiv:1903.12221 (2019).
[16]
Locust. [n. d.]. Locust: An open source load testing tool. https://locust.io/. [Last visited on September 28, 2020].
[17]
Johannes Manner, Martin Endreß, Tobias Heckel, and Guido Wirtz. 2018. Cold start influencing factors in function as a service. In 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion). IEEE, 181--188.
[18]
Garrett McGrath and Paul R Brenner. 2017. Serverless computing: Design, implementation, and performance. In 2017 IEEE 37th International Conference on Distributed Computing Systems Workshops (ICDCSW). IEEE, 405--410.
[19]
Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Naren Nayak, and Vadim Sukhomlinov. 2019. Agile cold starts for scalable serverless. In 11th {USENIX} Workshop on Hot Topics in Cloud Computing (HotCloud 19).
[20]
Stephen Soltesz, Herbert Pötzl, Marc E. Fiuczynski, Andy Bavier, and Larry Peterson. 2007. Container-Based Operating System Visualization: A Scalable, High-Performance Alternative to Hypervisors. SIGOPS Oper. Syst. Rev. 41, 3 (March 2007), 275--287.
[21]
Spring. [n. d.]. Getting Started | Building an Application with Spring Boot. https://spring.io/guides/gs/spring-boot/
[22]
Markus Thommes. 2017. Squeezing the milliseconds: How to make serverless platforms blazing fast! https://medium.com/openwhisk/squeezing-the-milliseconds-how-to-make-serverless-platforms-blazing-fast-aea0e9951bd0
[23]
Dominik Tornow. 2018. Imperative vs Declarative. https://medium.com/@dominik.tornow/imperative-vs-declarative-8abc7dcae82e
[24]
W. Wang, L. Zhang, D. Guo, S. Wu, H. Cui, and F. Bi. 2019. Reg: An Ultra-Lightweight Container That Maximizes Memory Sharing and Minimizes the Runtime Environment. In 2019 IEEE International Conference on Web Services (ICWS). 76--82.

Cited By

View all
  • (2024)Understanding Network Startup for Secure Containers in Multi-Tenant Clouds: Performance, Bottleneck and OptimizationProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688436(635-650)Online publication date: 4-Nov-2024
  • (2024)CPU throttling-aware AI-based autoscaling for Kubernetes2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)10.1109/PIMRC59610.2024.10817283(1-7)Online publication date: 2-Sep-2024
  • (2024)Comparative Analysis of Container Build Methods: A Performance Evaluation2024 47th MIPRO ICT and Electronics Convention (MIPRO)10.1109/MIPRO60963.2024.10569255(1960-1966)Online publication date: 20-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '21: Proceedings of the 36th Annual ACM Symposium on Applied Computing
March 2021
2075 pages
ISBN:9781450381048
DOI:10.1145/3412841
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. auto-scaling
  2. cold start
  3. container frameworks
  4. serverless

Qualifiers

  • Research-article

Conference

SAC '21
Sponsor:
SAC '21: The 36th ACM/SIGAPP Symposium on Applied Computing
March 22 - 26, 2021
Virtual Event, Republic of Korea

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)86
  • Downloads (Last 6 weeks)10
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Understanding Network Startup for Secure Containers in Multi-Tenant Clouds: Performance, Bottleneck and OptimizationProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688436(635-650)Online publication date: 4-Nov-2024
  • (2024)CPU throttling-aware AI-based autoscaling for Kubernetes2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)10.1109/PIMRC59610.2024.10817283(1-7)Online publication date: 2-Sep-2024
  • (2024)Comparative Analysis of Container Build Methods: A Performance Evaluation2024 47th MIPRO ICT and Electronics Convention (MIPRO)10.1109/MIPRO60963.2024.10569255(1960-1966)Online publication date: 20-May-2024
  • (2023)Visualizing Kubernetes Distributed Systems: An Exploratory Study2023 IEEE Working Conference on Software Visualization (VISSOFT)10.1109/VISSOFT60811.2023.00011(12-22)Online publication date: 1-Oct-2023
  • (2023)An Alternative to FaaS Cold Start Latency of Low Request Frequency Applications2023 3rd International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME)10.1109/ICECCME57830.2023.10253389(1-6)Online publication date: 19-Jul-2023
  • (2023)Image Preheating: Enables Fast Startup of Large-Scale Parallel Containerized Jobs2023 6th International Conference on Software Engineering and Computer Science (CSECS)10.1109/CSECS60003.2023.10428577(1-8)Online publication date: 22-Dec-2023
  • (2023)Zero-Cost In-Depth Enforcement of Network Policies for Low-Latency Cloud-Native Systems2023 IEEE 16th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD60044.2023.00036(249-261)Online publication date: Jul-2023
  • (2022)Custom Scheduling in Kubernetes: A Survey on Common Problems and Solution ApproachesACM Computing Surveys10.1145/354478855:7(1-37)Online publication date: 15-Dec-2022
  • (2022)Pervasive Container Computing–––a Fine-Grained Loosely Coupled Cloud Cluster2022 IEEE 21st International Conference on Ubiquitous Computing and Communications (IUCC/CIT/DSCI/SmartCNS)10.1109/IUCC-CIT-DSCI-SmartCNS57392.2022.00023(57-66)Online publication date: Dec-2022
  • (2022)SCAL-E: An Auto Scaling Agent for Optimum Big Data Load Balancing in Kubernetes Environments2022 International Conference on Computer, Information and Telecommunication Systems (CITS)10.1109/CITS55221.2022.9832990(1-5)Online publication date: 13-Jul-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media