Kim Long Ngo
Marin Litoiu
ABSTRACT
Function as a Service (FaaS) is a new software technology with promising features such as automated resource management and auto-scaling. Since these operational aspects are transparent, software engineers may not fully understand the scaling characteristics as well as limitations of this technology and this lack of information can lead to undesired performance results. To address these concerns, we perform a study to characterize FaaS' scalability with intensive workloads on three popular FaaS cloud platforms, namely Amazon AWS Lambda, IBM and Azure Cloud Function. We also study a workload smoother design pattern to examine if it enhances FaaS overall performance. The results show that different FaaS platforms adopt distinct scaling strategies and by applying a workload smoother, software engineers can achieve 99 - 100% success rates compared to 60 - 80% when FaaS' system is saturated.
- amazon.com. 2021. Best practices for organizing larger serverless applications. https://aws.amazon.com/blogs/compute/best-practices-for-organizinglarger-serverless-applications/. Last accessed: 01/25/2022.Google Scholar
- amazon.com. 2021. Building Applications with Serverless Architectures. https://aws.amazon.com/lambda/serverless-architectures-learn-more/. Last accessed: 09/08/2021.Google Scholar
- amazon.com. 2021. Lambda Function Scaling. https://docs.aws.amazon.com/ lambda/latest/dg/invocation-scaling.html. Last accessed: 09/13/2021.Google Scholar
- amazon.com. 2021. Understanding Container Reuse in AWS Lambda. https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/. Last accessed: 09/08/2021.Google Scholar
- apache.org. 2021. Apache JMeter. https://jmeter.apache.org/. Last accessed: 01/25/2022.Google Scholar
- Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM Symposium on Cloud Computing (Indianapolis, Indiana, USA) (SoCC '10). Association for Computing Machinery, New York, NY, USA, 143--154. https://doi.org/10.1145/1807128.1807152Google ScholarDigital Library
- Simon Eismann, Joel Scheuner, Erwin van Eyk, Maximilian Schwinger, Johannes Grohmann, Nikolas Herbst, Cristina L. Abad, and Alexandru Iosup. 2021. A Review of Serverless Use Cases and their Characteristics.Google Scholar
- Nikolas Roman Herbst, Samuel Kounev, and Ralf Reussner. 2013. Elasticity in Cloud Computing: What It Is, and What It Is Not. In 10th International Conference on Autonomic Computing (ICAC 13). USENIX Association, San Jose, CA, 23-- 27. https://www.usenix.org/conference/icac13/technical-sessions/presentation/herbstGoogle Scholar
- ibm.com. 2021. FaaS Principles and Best Practices. https://www.ibm.com/cloud/learn/faas. Last accessed: 01/25/2022.Google Scholar
- Zhen Ming Jiang and Ahmed E. Hassan. 2015. A Survey on Load Testing of Large-Scale Software Systems. IEEE Transactions on Software Engineering 41, 11 (2015), 1091--1118. https://doi.org/10.1109/TSE.2015.2445340Google ScholarDigital Library
- Jörn Kuhlenkamp, Sebastian Werner, Maria C. Borges, Dominik Ernst, and Daniel Wenzel. 2020. Benchmarking Elasticity of FaaS Platforms as a Foundation for Objective-Driven Design of Serverless Applications. In Proceedings of the 35th Annual ACM Symposium on Applied Computing (Brno, Czech Republic). Association for Computing Machinery, New York, NY, USA, 1576--1585. https://doi.org/10.1145/3341105.3373948Google ScholarDigital Library
- Sebastian Lehrig, Hendrik Eikerling, and Steffen Becker. 2015. Scalability, elasticity, and efficiency in cloud computing: A systematic literature review of definitions and metrics. In 2015 11th International ACM SIGSOFT Conference on Quality of Software Architectures (QoSA). 83--92. https://doi.org/10.1145/2737182.2737185Google ScholarDigital Library
- Wes Lloyd, Shruti Ramesh, Swetha Chinthalapati, Lan Ly, and Shrideep Pallickara. 2018. Serverless Computing: An Investigation of Factors Influencing Microservice Performance. In 2018 IEEE International Conference on Cloud Engineering (IC2E). 159--169.Google Scholar
- Horácio Martins, Filipe Araujo, and Paulo Rupino Cunha. 2020. Benchmarking Serverless Computing Platforms. Journal of Grid Computing 18 (12 2020), 691--709. https://doi.org/10.1007/s10723-020-09523--1Google Scholar
- microsoft.com. 2021. Azure Functions HTTP output bindings. https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindingshttp- webhook-output#hostjson-settings. Last accessed: 09/09/2021.Google Scholar
- microsoft.com. 2021. Best practices for performance and reliability of Azure Functions. https://docs.microsoft.com/en-us/azure/azure-functions/functionsbest- practices. Last accessed: 01/25/2022.Google Scholar
- microsoft.com. 2021. Cloud Design Patterns. https://docs.microsoft.com/enus/azure/architecture/patterns/. Last accessed: 09/30/2021.Google Scholar
- oracle.com. 2021. Java ThreadPoolExecutor. https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html. Last accessed: 09/26/2021.Google Scholar
- Andrei Palade, Aqeel Kazmi, and Siobhán Clarke. 2019. An Evaluation of Open Source Serverless Computing Frameworks Support at the Edge. In 2019 IEEE World Congress on Services (SERVICES), Vol. 2642--939X. 206--211. https://doi.org/10.1109/SERVICES.2019.00057Google ScholarCross Ref
- Joel Scheuner and Philipp Leitner. 2020. Function-as-a-Service performance evaluation: A multivocal literature review. Journal of Systems and Software 170 (2020), 110708. https://doi.org/10.1016/j.jss.2020.110708Google ScholarCross Ref
- Nikhila Somu, Nilanjan Daw, Umesh Bellur, and Purushottam Kulkarni. 2020. PanOpticon: A Comprehensive Benchmarking Tool for Serverless Applications. In 2020 International Conference on COMmunication Systems NETworkS (COMSNETS). 144--151. https://doi.org/10.1109/COMSNETS48256.2020.9027346Google Scholar
- spring.io. 2021. Spring Boot. https://spring.io/projects/spring-boot. Last accessed: 09/26/2021.Google Scholar
- Uma Tadakamalla and Daniel Menascé. 2019. Characterization of IoT Workloads. Springer International Publishing, 1--15. https://doi.org/10.1007/978--3- 030--23374--7_1Google Scholar
- Davide Taibi, Nabil El Ioini, Claus Pahl, and Jan Niederkofler. 2020. Patterns for Serverless Functions (Function-as-a-Service): A Multivocal Literature Review. In Proceedings of the 10th International Conference on Cloud Computing and Services Science - CLOSER,. https://doi.org/10.5220/0009578501810192Google ScholarCross Ref
- visualvm.github.io. 2021. VisualVM Tool. https://visualvm.github.io/. Last accessed: 01/25/2022.Google Scholar
- zenodo.org. 2021. Factorial Cloud Functions. https://doi.org/10.5281/zenodo.5865550. Last accessed: 12/23/2021.Google Scholar
Index Terms
- Evaluating the Scalability and Elasticity of Function as a Service Platform
Recommendations
Scalability, Elasticity, and Efficiency in Cloud Computing: a Systematic Literature Review of Definitions and Metrics
QoSA '15: Proceedings of the 11th International ACM SIGSOFT Conference on Quality of Software ArchitecturesContext: In cloud computing, there is a multitude of definitions and metrics for scalability, elasticity, and efficiency. However, stakeholders have little guidance for choosing fitting definitions and metrics for these quality properties, thus leading ...
Physics and microeconomics-based metrics for evaluating cloud computing elasticity
Currently, many customers and broadband providers are using cloud resources, such as processing and storage, for their applications and services. With the increase of computational resources usage, elasticity has become quite attractive and a key ...
A survey on elasticity management in PaaS systems
Elasticity is a goal of cloud computing. An elastic system should manage in an autonomic way its resources, being adaptive to dynamic workloads, allocating additional resources when workload is increased and deallocating resources when workload ...
Comments