research-article

pHPA: A Proactive Autoscaling Framework for Microservice Chain

Authors:

Byungkwon Choi,

Dongsu HanAuthors Info & Claims

APNet '21: Proceedings of the 5th Asia-Pacific Workshop on Networking

Pages 65 - 71

https://doi.org/10.1145/3469393.3469401

Published: 01 February 2022 Publication History

Abstract

Microservice is an architectural style that breaks down monolithic applications into smaller microservices and has been widely adopted by a variety of enterprises. Like the monolith, autoscaling has attracted the attention of operators in scaling microservices. However, most existing approaches of autoscaling do not consider microservice chain and severely degrade the performance of microservices when traffic surges. In this paper, we present pHPA, an autoscaling framework for the microservice chain. pHPA proactively allocates resources to the microservice chains and effectively handles traffic surges. Our evaluation using various open-source benchmarks shows that pHPA reduces 99%-tile latency and resource usage by up to 70% and 58% respectively compared to the most widely used autoscaler when traffic surges.

References

[1]

[n. d.]. Accelerate and Autoscale Deep Learning Inference on GPUs with KFServing. https://www.youtube.com/watch?v=LX_AWimNOhU. ([n. d.]).

[2]

[n. d.]. Auto Scaling Production Services on Titus. https://netflixtechblog.com/auto-scaling-production-services-on-titus-1f3cd49f5cd7. ([n. d.]).

[3]

[n. d.]. AWS Fargate. https://aws.amazon.com/fargate. ([n. d.]).

[4]

[n. d.]. Bookinfo Application by Istio. https://istio.io/latest/docs/examples/bookinfo/. ([n. d.]).

[5]

[n. d.]. Canada’s immigration website crashed due to traffic surge. https://www.ctvnews.ca/canada/canada-s-immigration-website-crashed-due-to-traffic-surge-1.3152744. ([n. d.]).

[6]

[n. d.]. Cloud Waste To Hit Over 14 Billion in 2019. https://devops.com/cloud-waste-to-hit-over-14-billion-in-2019/. ([n. d.]).

[7]

[n. d.]. Compiling and running a Java program. https://books.google.co.kr/books?id=8kD8j3FCk58C&pg=PR20. ([n. d.]).

[8]

[n. d.]. Configuring horizontal Pod autoscaling in GKE. https://cloud.google.com/kubernetes-engine/docs/how-to/horizontal-pod-autoscaling. ([n. d.]).

[9]

[n. d.]. Examples and types of microservices. https://www.itrelease.com/2018/10/examples-and-types-of-microservices/. ([n. d.]).

[10]

[n. d.]. Horizontal Pod Autoscaler in AWS. https://docs.aws.amazon.com/eks/latest/userguide/horizontal-pod-autoscaler.html. ([n. d.]).

[11]

[n. d.]. Internal documents show how Amazon scrambled to fix Prime Day glitches. https://www.cnbc.com/2018/07/19/amazon-internal-documents-what-caused-prime-day-crash-company-scramble.html. ([n. d.]).

[12]

[n. d.]. Jaeger: open source, end-to-end distributed tracing. https://www.jaegertracing.io/. ([n. d.]).

[13]

[n. d.]. Kubernetes: Production-Grade Container Orchestration. https://kubernetes.io/. ([n. d.]).

[14]

[n. d.]. Learning to Optimize with Reinforcement Learning. https://bair.berkeley.edu/blog/2017/09/12/learning-to-optimize-with-rl/. ([n. d.]).

[15]

[n. d.]. Linkerd: The world’s lightest, fastest service mesh.https://linkerd.io/. ([n. d.]).

[16]

[n. d.]. Locust: An open source load testing tool. https://locust.io/. ([n. d.]).

[17]

[n. d.]. Managing traffic spikes with schedule-based auto scaling. https://cloud.ibm.com/docs/virtual-servers?topic=virtual-servers-managing-schedule-based-auto-scaling. ([n. d.]).

[18]

[n. d.]. Microservice architecture growing in popularity, adopters enjoying success. https://www.itproportal.com/news/microservice-architecture-growing-in-popularity-adopters-enjoying-success/. ([n. d.]).

[19]

[n. d.]. Microservices - Netflix Techblog. https://netflixtechblog.com/tagged/microservices. ([n. d.]).

[20]

[n. d.]. Microservices Adoption in 2020. https://www.oreilly.com/radar/microservices-adoption-in-2020/. ([n. d.]).

[21]

[n. d.]. Microservices at Amazon. https://www.slideshare.net/apigee/i-love-apis-2015-microservices-at-amazon-54487258. ([n. d.]).

[22]

[n. d.]. Microsoft Confirms March Azure Outage Due to COVID-19 Strains. https://visualstudiomagazine.com/articles/2020/04/13/azure-outage.aspx. ([n. d.]).

[23]

[n. d.]. Multi-Tenancy Kubernetes on Bare Metal Servers. https://deview.kr/data/deview/2019/presentation/[231]+Multi-Tenancy+Kubernetes+on+Bare+Metal+Servers.pdf(16p). ([n. d.]).

[24]

[n. d.]. Online Boutique by Google. https://github.com/GoogleCloudPlatform/microservices-demo. ([n. d.]).

[25]

[n. d.]. Prometheus - Monitoring system & time series database. https://prometheus.io/. ([n. d.]).

[26]

[n. d.]. Scale applications in Azure Kubernetes Service (AKS). https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-scale. ([n. d.]).

[27]

[n. d.]. Scaling based on schedules. https://cloud.google.com/compute/docs/autoscaler/scaling-schedules. ([n. d.]).

[28]

[n. d.]. Schedule-Based Autoscaling. https://docs.oracle.com/en-us/iaas/Content/Compute/Tasks/autoscalinginstancepools.htm#time. ([n. d.]).

[29]

[n. d.]. Scheduled scaling for Amazon EC2 Auto Scaling. https://docs.aws.amazon.com/autoscaling/ec2/userguide/schedule_time.html. ([n. d.]).

[30]

[n. d.]. Smooth maximum. https://en.wikipedia.org/wiki/Smooth_maximum. ([n. d.]).

[31]

[n. d.]. Stan’s Robot Shop by Instana. https://www.instana.com/blog/stans-robot-shop-sample-microservice-application/. ([n. d.]).

[32]

[n. d.]. The Story of Netflix and Microservices. https://www.geeksforgeeks.org/the-story-of-netflix-and-microservices/. ([n. d.]).

[33]

[n. d.]. Throughput Autoscaling: Dynamic sizing for facebook.com. https://www.facebook.com/watch/?v=378764619812456. ([n. d.]).

[34]

[n. d.]. Understanding Container Reuse in AWS Lambda. https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/. ([n. d.]).

[35]

[n. d.]. Variational and Approximate GPs. https://docs.gpytorch.ai/en/v1.1.1/examples/04_Variational_and_Approximate_GPs/. ([n. d.]).

[36]

[n. d.]. Vegeta: a versatile HTTP load testing tool. https://github.com/tsenart/vegeta. ([n. d.]).

[37]

[n. d.]. Wasted Cloud Spend to Exceed 17.6 Billion in 2020, Fueled by Cloud Computing Growth. https://jaychapel.medium.com/wasted-cloud-spend-to-exceed-17-6-billion-in-2020-fueled-by-cloud-computing-growth-7c8f81d5c616. ([n. d.]).

[38]

[n. d.]. Zoom suffers ”partial outage” amid home working surge. https://www.datacenterdynamics.com/en/news/zoom-suffers-partial-outage-amid-home-working-surge/. ([n. d.]).

[39]

[n. d.]. “Google Down”: How Users Experienced Google’s Major Outage. https://www.semrush.com/blog/google-down-how-users-experienced-google-major-outage/. ([n. d.]).

[40]

2021. Horizontal Pod Autoscaler of Kubernetes. https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/. (2021).

[41]

Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. 2016. Learning to learn by gradient descent by gradient descent. arXiv preprint arXiv:1606.04474(2016).

[42]

Melanie Cebula. 2017. Airbnb, From Monolith to Microservices: How to Scale Your Architecture. https://www.youtube.com/watch?v=N1BWMW9NEQc. (2017).

[43]

Hanjun Dai, Elias B Khalil, Yuyu Zhang, Bistra Dilkina, and Le Song. 2017. Learning combinatorial optimization algorithms over graphs. arXiv preprint arXiv:1704.01665(2017).

[44]

Silvery Fu, Radhika Mittal, Lei Zhang, and Sylvia Ratnasamy. 2020. Fast and efficient container startup at the edge via dependency scheduling. In 3rd {USENIX} Workshop on Hot Topics in Edge Computing (HotEdge 20).

[45]

Yu Gan, Yanqi Zhang, Dailun Cheng, Ankitha Shetty, Priyal Rathi, Nayan Katarki, Ariana Bruno, Justin Hu, Brian Ritchken, Brendon Jackson, 2019. An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. 3–18.

Digital Library

[46]

Alim Ul Gias, Giuliano Casale, and Murray Woodside. 2019. ATOM: Model-driven autoscaling for microservices. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). IEEE, 1994–2004.

[47]

Tyler Harter, Brandon Salmon, Rose Liu, Andrea C Arpaci-Dusseau, and Remzi H Arpaci-Dusseau. 2016. Slacker: Fast distribution with lazy docker containers. In 14th {USENIX} Conference on File and Storage Technologies ({FAST} 16). 181–195.

[48]

Martin Jankowiak, Geoff Pleiss, and Jacob R. Gardner. 2020. Parametric Gaussian Process Regressors. In Proceedings of the 37th International Conference on Machine Learning, Online, PMLR.

Digital Library

[49]

Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the cloud: Distributed computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing. 445–451.

Digital Library

[50]

Ke Li and Jitendra Malik. 2016. Learning to optimize. arXiv preprint arXiv:1606.01885(2016).

[51]

Zhuwen Li, Qifeng Chen, and Vladlen Koltun. 2018. Combinatorial optimization with graph convolutional networks and guided tree search. arXiv preprint arXiv:1810.10659(2018).

[52]

Hongzi Mao, Malte Schwarzkopf, Shaileshh Bojja Venkatakrishnan, Zili Meng, and Mohammad Alizadeh. 2019. Learning scheduling algorithms for data processing clusters. In Proceedings of the ACM Special Interest Group on Data Communication. 270–288.

Digital Library

[53]

Haoran Qiu, Subho S. Banerjee, Saurabh Jha, Zbigniew T. Kalbarczyk, and Ravishankar K. Iyer. 2020. FIRM: An Intelligent Fine-grained Resource Management Framework for SLO-Oriented Microservices. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). USENIX Association, 805–825. https://www.usenix.org/conference/osdi20/presentation/qiu

[54]

Charles Reiss, Alexey Tumanov, Gregory R Ganger, Randy H Katz, and Michael A Kozuch. 2012. Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In Proceedings of the third ACM symposium on cloud computing. 1–13.

Digital Library

[55]

Krzysztof Rzadca, Paweł Findeisen, Jacek Świderski, Przemyslaw Zych, Przemyslaw Broniek, Jarek Kusmierek, Paweł Krzysztof Nowak, Beata Strack, Piotr Witusowski, Steven Hand, and John Wilkes. 2020. Autopilot: Workload Autoscaling at Google Scale. In Proceedings of the Fifteenth European Conference on Computer Systems. https://dl.acm.org/doi/10.1145/3342195.3387524

Digital Library

[56]

Abhishek Verma, Luis Pedrosa, Madhukar R. Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. 2015. Large-scale cluster management at Google with Borg. In Proceedings of the European Conference on Computer Systems (EuroSys). Bordeaux, France.

Digital Library

[57]

Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking behind the curtains of serverless platforms. In 2018 {USENIX} Annual Technical Conference ({USENIX}{ATC} 18). 133–146.

[58]

Z. Yang, P. Nguyen, H. Jin, and K. Nahrstedt. 2019. MIRAS: Model-based Reinforcement Learning for Microservice Resource Allocation over Scientific Workflows. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS). 122–132. https://doi.org/10.1109/ICDCS.2019.00021

[59]

Guangba Yu, Pengfei Chen, and Zibin Zheng. 2019. Microscaler: Automatic scaling for microservices with an online learning approach. In 2019 IEEE International Conference on Web Services (ICWS). IEEE, 68–75.

[60]

Nannan Zhao, Vasily Tarasov, Hadeel Albahar, Ali Anwar, Lukas Rupprecht, Dimitrios Skourtis, Amit S Warke, Mohamed Mohamed, and Ali R Butt. 2019. Large-scale analysis of the docker hub dataset. In 2019 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 1–10.

Cited By

Wang ZLi PLiang CWu FYan FVanbever LZhang I(2024)AutothrottleProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691834(149-165)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691834
Yuan HLiao S(2024)A Time Series-Based Approach to Elastic Kubernetes ScalingElectronics10.3390/electronics1302028513:2(285)Online publication date: 8-Jan-2024
https://doi.org/10.3390/electronics13020285
Park JPark JJung YLim HYeo HHan DSekar VYu MSeneviratne AVeitch D(2024)TopFull: An Adaptive Top-Down Overload Control for SLO-Oriented MicroservicesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672253(876-890)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672253
Show More Cited By

Index Terms

pHPA: A Proactive Autoscaling Framework for Microservice Chain

Index terms have been assigned to the content through auto-classification.

Recommendations

Practical Efficient Microservice Autoscaling with QoS Assurance
HPDC '22: Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing

Cloud applications are increasingly moving away from monolithic services to agile microservices-based deployments. However, efficient resource management for microservices poses a significant hurdle due to the sheer number of loosely coupled and ...
ProScale: Proactive Autoscaling for Microservice With Time-Varying Workload at the Edge
Deploying microservice instances on the edge device close to end users can provide on-site processing thus reducing request response time. Each microservice has multiple instances that can process requests in parallel. To achieve high processing ...
Proactive–reactive microservice architecture global scaling
Abstract
We develop a novel approach for run-time global adaptation of microservice applications, based on synthesis of architecture-level reconfigurations. More precisely, we devise an algorithm for proactive–reactive automatic scaling that reaches a ...
Highlights
- Novel microservice architectures scaling approach improving the state-of-the-art.
- New algorithm for proactive–reactive global scaling.
- Novel integrated timed architectural modeling/execution language.
- Benchmarks evaluating the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

APNet '21: Proceedings of the 5th Asia-Pacific Workshop on Networking

June 2021

79 pages

ISBN:9781450385879

DOI:10.1145/3469393

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 February 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Toyota Motor Corporation

Conference

APNet 2021

APNet 2021: 5th Asia-Pacific Workshop on Networking

June 24 - 25, 2021

Shenzhen, China, China

Acceptance Rates

Overall Acceptance Rate 50 of 118 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
324
Total Downloads

Downloads (Last 12 months)94
Downloads (Last 6 weeks)7

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang ZLi PLiang CWu FYan FVanbever LZhang I(2024)AutothrottleProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691834(149-165)Online publication date: 16-Apr-2024
https://dl.acm.org/doi/10.5555/3691825.3691834
Yuan HLiao S(2024)A Time Series-Based Approach to Elastic Kubernetes ScalingElectronics10.3390/electronics1302028513:2(285)Online publication date: 8-Jan-2024
https://doi.org/10.3390/electronics13020285
Park JPark JJung YLim HYeo HHan DSekar VYu MSeneviratne AVeitch D(2024)TopFull: An Adaptive Top-Down Overload Control for SLO-Oriented MicroservicesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672253(876-890)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672253
Xie SWang JLi BZhang ZLi DHung P(2024)PBScaler: A Bottleneck-Aware Autoscaling Framework for Microservice-Based ApplicationsIEEE Transactions on Services Computing10.1109/TSC.2024.337620217:2(604-616)Online publication date: Mar-2024
https://doi.org/10.1109/TSC.2024.3376202
Park JChoi BLee CHan D(2024)Graph Neural Network-Based SLO-Aware Proactive Resource Autoscaling Framework for MicroservicesIEEE/ACM Transactions on Networking10.1109/TNET.2024.339342732:4(3331-3346)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3393427
Ahmad HTreude CWagner MSzabo C(2024)Smart HPA: A Resource-Efficient Horizontal Pod Auto-Scaler for Microservice Architectures2024 IEEE 21st International Conference on Software Architecture (ICSA)10.1109/ICSA59870.2024.00013(46-57)Online publication date: 4-Jun-2024
https://doi.org/10.1109/ICSA59870.2024.00013
Kumar BVerma AVerma P(2024)Optimizing resource allocation using proactive scaling with predictive models and custom resourcesComputers and Electrical Engineering10.1016/j.compeleceng.2024.109419118(109419)Online publication date: Sep-2024
https://doi.org/10.1016/j.compeleceng.2024.109419
Soldani JForti SRoveroni LBrogi A(2024)Explaining Microservices' Cascading Failures From Their LogsSoftware: Practice and Experience10.1002/spe.3400Online publication date: 17-Dec-2024
https://doi.org/10.1002/spe.3400
Cheng KZhang STu CShi XYin ZLu SLiang YGu Q(2023)ProScale: Proactive Autoscaling for Microservice With Time-Varying Workload at the EdgeIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2023.323842934:4(1294-1312)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TPDS.2023.3238429
Bhardwaj A(2023)Navigating the Complexities of the Cloud-Native World: A Study of Developer Perspectives2023 6th International Conference on Advanced Communication Technologies and Networking (CommNet)10.1109/CommNet60167.2023.10365297(1-6)Online publication date: 11-Dec-2023
https://doi.org/10.1109/CommNet60167.2023.10365297
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten