Hanfei Yu
ABSTRACT
Serverless computing has been favored by users and infrastructure providers from various industries, including online services and scientific computing. Users enjoy its auto-scaling and ease-of-management, and providers own more control to optimize their service. However, existing serverless platforms still require users to pre-define resource allocations for their functions, leading to frequent misconfiguration by inexperienced users in practice. Besides, functions' varying input data further escalate the gap between their dynamic resource demands and static allocations, leaving functions either over-provisioned or under-provisioned. This paper presents Libra, a safe and timely resource harvesting framework for multi-node serverless clusters. Libra makes precise harvesting decisions to accelerate function invocations with harvested resources and jointly improve resource utilization by profiling dynamic resource demands and availability proactively. Experiments on OpenWhisk clusters with real-world workloads show that Libra reduces response latency by 39% and achieves 3X resource utilization compared to state-of-the-art solutions.
- Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, and Sudheendra Vijayanarasimhan. 2016. Youtube-8m: A Large-scale Video Classification Benchmark. arXiv preprint arXiv:1609.08675 (2016).Google Scholar
- Nabeel Akhtar, Ali Raza, Vatche Ishakian, and Ibrahim Matta. 2020. COSE: Configuring Serverless Functions using Statistical Learning. In Proc. of the 2020 IEEE Conference on Computer Communications (INFOCOM).Google ScholarDigital Library
- Ahsan Ali, Riccardo Pinciroli, Feng Yan, and Evgenia Smirni. 2020. BATCH: Machine Learning Inference Serving on Serverless Platforms With Adaptive Batching. In Proc. of the IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC). IEEE, 1--15.Google ScholarCross Ref
- Pradeep Ambati, Í nigo Goiri, Felipe Frujeri, Alper Gun, Ke Wang, Brian Dolan, Brian Corell, Sekhar Pasupuleti, Thomas Moscibroda, Sameh Elnikety, et al. 2020. Providing SLOs for Resource-Harvesting VMs in Cloud Platforms. In Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation (OSDI).Google ScholarDigital Library
- Lixiang Ao, Liz Izhikevich, Geoffrey M Voelker, and George Porter. 2018. Sprocket: A Serverless Video Processing Framework. In Proc. of the ACM Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- Apache. 2018. Apache OpenWhisk: Open Source Serverless Cloud Platform. https://openwhisk.apache.org. [Online; accessed 1-May-2018].Google Scholar
- Arda Aytekin and Mikael Johansson. 2019. Harnessing the Power of Serverless Runtimes for Large-Scale Optimization. arXiv preprint arXiv:1901.03161 (2019).Google Scholar
- Yossi Azar and Danny Vainstein. 2019. Tight Bounds for Clairvoyant Dynamic Bin Packing. ACM Transactions on Parallel Computing (TOPC) (2019).Google Scholar
- Bharathan Balaji, Christopher Kakovitch, and Balakrishnan Narayanaswamy. 2021. FirePlace: Placing Firecraker Virtual Machines with Hindsight Imitation. Proc. of Machine Learning and Systems (MLSys), Vol. 3 (2021).Google Scholar
- Vivek M Bhasi, Jashwant Raj Gunasekaran, Aakash Sharma, Mahmut Taylan Kandemir, and Chita Das. 2022. Cypress: Input Size-sensitive Container Provisioning and Request Scheduling for Serverless Platforms. In Proceedings of the 13th Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz. 2019. Cirrus: a Serverless Framework for End-to-end ML Workflows. In Proc. of the ACM Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, and Kyle Chard. 2020. FuncX: A Federated Function Serving Fabric for Science. In Proc. of The 29th International Symposium on High-Performance Parallel and Distributed Computing (HPDC). 65--76.Google ScholarDigital Library
- Edward G Coffman, Jr, Michael R Garey, and David S Johnson. 1983. Dynamic Bin Packing. SIAM J. Comput. (1983).Google Scholar
- Marcin Copik et al. 2020. SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing. arXiv preprint arXiv:2012.14132 (2020).Google Scholar
- Gabor Csardi, Tamas Nepusz, et al. 2006. The igraph Software Package for Complex Network Research. InterJournal, complex systems (2006).Google Scholar
- Christina Delimitrou and Christos Kozyrakis. 2014. Quasar: Resource-efficient and QoS-aware Cluster Management. ACM SIGPLAN Notices (2014).Google ScholarDigital Library
- Docker. 2021. Docker Update API. https://docs.docker.com/engine/reference/commandline/update/. [Online].Google Scholar
- Simon Eismann, Long Bui, Johannes Grohmann, Cristina Abad, Nikolas Herbst, and Samuel Kounev. 2021. Sizeless: Predicting the Optimal Size of Serverless Functions. In Proc. of the 22nd International Middleware Conference (MIDDLEWARE).Google ScholarDigital Library
- Sadjad Fouladi, Riad S Wahby, Brennan Shacklett, Karthikeyan Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In Proc. of USENIX NSDI.Google Scholar
- Alexander Fuerst, Stanko Novaković, Í nigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini. 2022. Memory-harvesting VMs in Cloud Platforms. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS).Google ScholarDigital Library
- Alexander Fuerst and Prateek Sharma. 2022. Locality-aware Load-Balancing For Serverless Clusters. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing (HPDC).Google ScholarDigital Library
- Jashwant Raj Gunasekaran, Prashanth Thinakaran, Nachiappan Chidambaram, Mahmut T Kandemir, and Chita R Das. 2020. Fifer: Tackling Underutilization in the Serverless Era. In Proc. the 21st International Middleware Conference (Middleware).Google ScholarDigital Library
- Varun Gupta, Mor Harchol Balter, Karl Sigman, and Ward Whitt. 2007. Analysis of Join-the-Shortest-Queue Routing for Web Server Farms. Performance Evaluation (2007).Google Scholar
- Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, et al. 2020. Array Programming with NumPy. Nature (2020).Google Scholar
- Heo, Tejun. 2021. Control Group v2. https://www.kernel.org/doc/Documentation/cgroup-v2.txt. [Online; accessed 1-April-2022].Google Scholar
- Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning Multiple Layers of Features from Tiny Images. (2009).Google Scholar
- Ashraf Mahgoub, Li Wang, Karthick Shankar, Yiming Zhang, Huangshi Tian, Subrata Mitra, Yuxing Peng, Hongqi Wang, Ana Klimovic, Haoran Yang, et al. 2021. SONIC: Application-aware Data Passing for Chained Serverless Applications. In Proc. of the 2021 USENIX Annual Technical Conference (USENIX ATC).Google Scholar
- Bilal Muhammad, Canini Marco, Fonseca Rodrigo, and Rodrigues Rodrigo. 2023. With Great Freedom Comes Great Opportunity: Rethinking Resource Allocation for Serverless Functions. In Proceedings of the European Conference on Computer Systems (EuroSys).Google Scholar
- Djob Mvondo, Mathieu Bacou, Kevin Nguetchouang, Lucien Ngale, Stéphane Pouget, Josiane Kouam, Renaud Lachaize, Jinho Hwang, Tim Wood, Daniel Hagimont, et al. 2021. OFC: an Opportunistic Caching System for FaaS Platforms. In Proceedings of the Sixteenth European Conference on Computer Systems (EuroSys).Google ScholarDigital Library
- Jinwoo Park, Byungkwon Choi, Chunghan Lee, and Dongsu Han. 2021. GRAF: A Graph Neural Network Based Proactive Resource Allocation Framework for SLO-Oriented Microservices. In Proc. of the 17th International Conference on emerging Networking EXperiments and Technologies (CONEXT).Google ScholarDigital Library
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research (JMLR) (2011).Google Scholar
- Benjamin Reidys, Jinghan Sun, Anirudh Badam, Shadi Noghabi, and Jian Huang. 2022. BlockFlex: Enabling Storage Harvesting with Software-Defined Flash in Modern Cloud Platforms. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI).Google Scholar
- Francisco Romero, Gohar Irfan Chaudhry, Í nigo Goiri, Pragna Gopa, Paul Batum, Neeraja J Yadwadkar, Rodrigo Fonseca, Christos Kozyrakis, and Ricardo Bianchini. 2021. Faa$T: A Transparent Auto-Scaling Cache for Serverless Applications. In Proceedings of the ACM Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- Rohan Basu Roy, Tirthak Patel, Vijay Gadepally, and Devesh Tiwari. 2022. Mashup: Making Serverless Computing Useful for HPC Workflows via Hybrid Execution. In Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP).Google ScholarDigital Library
- Mohammad Shahrad, Jonathan Balkind, and David Wentzlaff. 2019. Architectural Implications of Function-as-a-Service Computing. In Proc. of IEEE/ACM MICRO.Google ScholarDigital Library
- Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In Proc. of USENIX ATC.Google Scholar
- Vaishaal Shankar, Karl Krauth, Kailas Vodrahalli, Qifan Pu, Benjamin Recht, Ion Stoica, Jonathan Ragan-Kelley, Eric Jonas, and Shivaram Venkataraman. 2020. Serverless Linear Algebra. In Proc. of the 11th ACM Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- Arjun Singhvi, Arjun Balasubramanian, Kevin Houck, Mohammed Danish Shaikh, Shivaram Venkataraman, and Aditya Akella. 2021. Atoll: A Scalable Low-Latency Serverless Platform. In Proc. of the ACM Symposium on Cloud Computing (SoCC). 138--152.Google ScholarDigital Library
- Tyler J Skluzacek, Ryan Wong, Zhuozhao Li, Ryan Chard, Kyle Chard, and Ian Foster. 2021. A Serverless Framework for Distributed Bulk Metadata Extraction. In Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing (HPDC).Google ScholarDigital Library
- Craig A Stewart, Timothy M Cockerill, Ian Foster, David Hancock, Nirav Merchant, Edwin Skidmore, Daniel Stanzione, James Taylor, Steven Tuecke, George Turner, et al. 2015. Jetstream: a Self-provisioned, Scalable Science and Engineering Cloud Environment. In Proc. of the 2015 XSEDE Conference: Scientific Advancements Enabled by Enhanced Cyberinfrastructure (XSEDE).Google ScholarDigital Library
- Amoghavarsha Suresh and Anshul Gandhi. 2021. ServerMore: Opportunistic Execution of Serverless Functions in the Cloud. In Proc. of the ACM Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- Huangshi Tian, Suyi Li, Ao Wang, Wei Wang, Tianlong Wu, and Haoran Yang. 2022. Owl: Performance-aware Scheduling for Resource-efficient Function-as-a-Service Cloud. In Proc. of the 13th ACM Symposium on Cloud Computing (SoCC).Google ScholarDigital Library
- John Towns, Timothy Cockerill, Maytal Dahan, Ian Foster, Kelly Gaither, Andrew Grimshaw, Victor Hazlewood, Scott Lathrop, Dave Lifka, Gregory D Peterson, et al. 2014. XSEDE: Accelerating Scientific Discovery. Computing in Science & Engineering (2014).Google Scholar
- Hao Wang, Di Niu, and Baochun Li. 2019. Distributed Machine Learning with a Serverless Architecture. In Proc. of IEEE INFOCOM.Google ScholarDigital Library
- Yawen Wang, Kapil Arya, Marios Kogias, Manohar Vanga, Aditya Bhandari, Neeraja J Yadwadkar, Siddhartha Sen, Sameh Elnikety, Christos Kozyrakis, and Ricardo Bianchini. 2021. SmartHarvest: Harvesting Idle CPUs Safely and Efficiently in the Cloud. In Proc. of ACM EuroSys.Google ScholarDigital Library
- Zhaojie Wen, Yishuo Wang, and Fangming Liu. 2022. StepConf: SLO-Aware Dynamic Resource Configuration for Serverless Function Workflows. In Proc. of the IEEE International Conference on Computer Communications (INFOCOM).Google ScholarDigital Library
- David L Wheeler, Tanya Barrett, Dennis A Benson, Stephen H Bryant, Kathi Canese, Vyacheslav Chetvernin, Deanna M Church, Michael DiCuccio, Ron Edgar, Scott Federhen, et al. 2007. Database Resources of the National Center for Biotechnology Information. Nucleic Acids Research (2007).Google Scholar
- Hanfei Yu, Athirai A Irissappane, Hao Wang, and Wes J Lloyd. 2021. FaaSRank: Learning to Schedule Functions in Serverless Platforms. In Proc. of IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS).Google ScholarCross Ref
- Hanfei Yu, Hao Wang, Jian Li, Xu Yuan, and Seung-Jong Park. 2022. Accelerating Serverless Computing by Harvesting Idle Resources. In Proc. of the ACM Web Conference (WWW).Google ScholarDigital Library
- Yanqi Zhang, Í nigo Goiri, Gohar Irfan Chaudhry, Rodrigo Fonseca, Sameh Elnikety, Christina Delimitrou, and Ricardo Bianchini. 2021. Faster and Cheaper Serverless Computing on Harvested Resources. In Proc. of the ACM SIGOPS 28th Symposium on Operating Systems Principles (SOSP).Google ScholarDigital Library
Index Terms
- Libra: Harvesting Idle Resources Safely and Timely in Serverless Clusters
Recommendations
Accelerating Serverless Computing by Harvesting Idle Resources
WWW '22: Proceedings of the ACM Web Conference 2022Serverless computing automates fine-grained resource scaling and simplifies the development and deployment of online services with stateless functions. However, it is still non-trivial for users to allocate appropriate resources due to various function ...
Faster and Cheaper Serverless Computing on Harvested Resources
SOSP '21: Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems PrinciplesServerless computing is becoming increasingly popular due to its ease of programming, fast elasticity, and fine-grained billing. However, the serverless provider still needs to provision, manage, and pay the IaaS provider for the virtual machines (VMs) ...
Supporting Multi-Provider Serverless Computing on the Edge
ICPP Workshops '18: Workshop Proceedings of the 47th International Conference on Parallel ProcessingServerless computing has recently emerged as a new execution model for cloud computing, in which service providers offer compute runtimes, also known as Function-as-a-Service (FaaS) platforms, allowing users to develop, execute and manage application ...
Comments