Skip to main content
SpringerLink
Log in

Kubernetes distributions for the edge: serverless performance evaluation

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Serverless computing, especially when deployed at the edge of the network, is seen as an enabling technology for the future development of more complex Internet of Things systems. However, special care must be taken when deploying new edge infrastructures for serverless workloads in terms of resource usage and network connectivity. Inefficient utilization of the available computing resources might easily cancel out the benefits acquired by moving the equipment closer to the edge, namely the reduced communication latency. Containers, together with the Kubernetes container orchestrator, are used by many serverless platforms today. We evaluate the performance of three different Kubernetes distributions—full-fledged Kubernetes, K3s, and MicroK8s when deployed in a resource constrained environment at the edge. We use the OpenFaaS serverless platform and employ 14 different benchmarks divided into three separate categories to evaluate various aspects of the execution performance of the distributions. Four different test types are performed focusing on cold start latency, serial execution performance, parallel execution using a single replica, and parallel execution utilizing different autoscaling strategies. Our results show that the edge-oriented K3s and MicroK8s distributions offer better performance in the majority of the tests, while a full-fledged deployment exhibits noticeable advantages for sustained loads such as parallel function invocation using a single replica.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. The extreme outliers for the K3s platforms (larger than 17s) have been cut off to aid the visibility of the figure. However, they have been taken into account in all other analyses and calculations.

References

  1. Mell P, Grance T (2011) The NIST definition of cloud computing. Technical report NIST special publication (SP) 800-145, National institute of standards and technology. https://doi.org/10.6028/NIST.SP.800-145

  2. Armbrust M, Fox A, Griffith R, Joseph A, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I, Zaharia M (2009) Above the clouds, A Berkeley view of cloud computing. http://bnrg.eecs.berkeley.edu/randy/Papers/RHKPubs07-11/A. Accessed 26 Dec 2021

  3. Duan Y, Fu G, Zhou N, Sun X, Narendra NC, Hu B (2015) Everything as a service (XaaS) on the cloud, origins current and future trends. In: 2015 IEEE 8th International Conference on Cloud Computing, pp 621–628. https://doi.org/10.1109/CLOUD.2015.88

  4. Jonas E, Schleier-Smith J, Sreekanti V, Tsai C.-C, Khandelwal A, Pu Q, Shankar V, Carreira J, Krauth K, Yadwadkar N, Gonzalez J.E, Popa R.A, Stoica I, Patterson DA (2019) Cloud programming simplified, A Berkeley view on serverless computing. arXiv:1902.03383 [cs]

  5. Kratzke N (2018) A brief history of cloud application architectures. Appl Sci 8(8):1368. https://doi.org/10.3390/app8081368

    Article  Google Scholar 

  6. Eismann S, Scheuner J, van Eyk E, Schwinger M, Grohmann J, Herbst N, Abad CL, Iosup A (2021) Serverless applications, why when, and how? IEEE Softw 38(1):32–39. https://doi.org/10.1109/MS.2020.3023302

    Article  Google Scholar 

  7. Bittencourt L, Immich R, Sakellariou R, Fonseca N, Madeira E, Curado M, Villas L, DaSilva L, Lee C, Rana O (2018) The Internet of Things fog and cloud continuum: integration and challenges. Internet of Things 3–4:134–155. https://doi.org/10.1016/j.iot.2018.09.005

    Article  Google Scholar 

  8. Varghese B, Buyya R (2018) Next generation cloud computing: new trends and research directions. Future Gener Comput Syst 79:849–861. https://doi.org/10.1016/j.future.2017.09.020

    Article  Google Scholar 

  9. Pfandzelter T, Bermbach D (2019)IoT data processing in the fog, functions streams, or batch processing? In: 2019 IEEE International Conference on Fog Computing (ICFC), pp 201–206. IEEE Prague, Czech Republic. https://doi.org/10.1109/ICFC.2019.00033

  10. Carvalho G, Cabral B, Pereira V, Bernardino J (2021) Edge computing: current trends, research challenges and future directions. Computing 103(5):993–1023. https://doi.org/10.1007/s00607-020-00896-5

    Article  Google Scholar 

  11. AWS IoT Greengrass: Amazon Web Services. https://aws.amazon.com/greengrass/ Accessed 26 April 2021

  12. IoT Hub | Microsoft Azure. https://azure.microsoft.com/en-us/services/iot-hub/ Accessed 26 April 2021

  13. Das A, Patterson S, Wittie M (2018) EdgeBench, benchmarking edge computing platforms. In: 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion), pp 175–180. IEEE Zurich. https://doi.org/10.1109/UCC-Companion.2018.00053

  14. Li J, Kulkarni S.G, Ramakrishnan KK, Li D (2021) Analyzing open-source serverless platforms, characteristics and performance, pp 15–20. https://doi.org/10.18293/SEKE2021-129

  15. Kjorveziroski V, Canto CB, Roig PJ, Gilly K, Mishev A, Trajkovik V, Filiposka S (2021) IoT serverless computing at the edge, open issues and research direction. Trans Netw Commun 9(4):1–33. https://doi.org/10.14738/tnc.94.11231

    Article  Google Scholar 

  16. Gadepalli P.K, McBride S, Peach G, Cherkasova L, Parmer G (2020) Sledge: a Serverless-first, Light-weight Wasm Runtime for the Edge. In: Proceedings of the 21st International Middleware Conference. Middleware ’20, pp 265–279. Association for Computing Machinery New York. https://doi.org/10.1145/3423211.3425680

  17. 2021 Kubernetes Adoption Survey. https://www.purestorage.com/content/dam/pdf/en/analyst-reports/ar-portworx-pure-storage-2021-kubernetes-adoption-survey.pdf Accessed 26 Dec 2021

  18. Kjorveziroski V, Filiposka S, Trajkovik V (2021) IoT serverless computing at the edge. A systematic mapping review. Computers 10(10):130. https://doi.org/10.3390/computers10100130

    Article  Google Scholar 

  19. Risco S, Moltó G, Naranjo DM, Blanquer I (2021) Serverless workflows for containerised applications in the cloud continuum. J Grid Comput 19(3):30. https://doi.org/10.1007/s10723-021-09570-2

    Article  Google Scholar 

  20. Kayal P (2020) Kubernetes in fog computing, feasibility demonstration limitations and improvement scope: invited paper. In: 2020 IEEE 6th World Forum on Internet of Things (WF-IoT), pp 1–6. https://doi.org/10.1109/WF-IoT48130.2020.9221340

  21. K3s: Lightweight Kubernetes. https://k3s.io/ Accessed 05 Sept 2021

  22. MicroK8s-Zero-ops Kubernetes for developers, edge and IoT | MicroK8s. http://microk8s.io Accessed 05 Sept 2021

  23. Software conformance(Certified Kubernetes). https://www.cncf.io/certification/software-conformance/ Accessed 26 Dec 2021

  24. Martins H, Araujo F, da Cunha PR (2020) Benchmarking serverless computing platforms. J Grid Comput 18(4):691–709. https://doi.org/10.1007/s10723-020-09523-1

    Article  Google Scholar 

  25. Gan Y, Zhang Y, Cheng D, Shetty A, Rathi P, Katarki N, Bruno A, Hu J, Ritchken B, Jackson B, Hu K, Pancholi M, He Y, Clancy B, Colen C, Wen F, Leung C, Wang S, Zaruvinsky L, Espinosa M, Lin R, Liu Z, Padilla J, Delimitrou C (2019) An open-source benchmark suite for microservices and their hardware-software implications for cloud & edge systems. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. ASPLOS ’19, pp 3–18. Association for Computing Machinery New York. https://doi.org/10.1145/3297858.3304013

  26. Wen J, Liu Y, Chen Z, Chen J, Ma Y (2021) Characterizing commodity serverless computing platforms. J Softw: Evol Process. https://doi.org/10.1002/smr.2394

    Article  Google Scholar 

  27. Scheuner J, Leitner P (2020) Function-as-a-service performance evaluation: a multivocal literature review. J Syst Softw. https://doi.org/10.1016/j.jss.2020.110708

    Article  Google Scholar 

  28. Hellerstein JM, Faleiro J, Gonzalez JE, Schleier-Smith J, Sreekanti V, Tumanov A, Wu C (2018) Serverless computing, one step forward two steps back. In: CIDR 20019 Monterey, CA. http://cidrdb.org/cidr2019/papers/p119-hellerstein-cidr19.pdf Accessed 09 Jan 2022

  29. Maissen P, Felber P, Kropf P, Schiavoni V (2020) FaaSdom, a benchmark suite for serverless computing. In: Proceedings of the 14th ACM International Conference on Distributed and Event-based Systems, pp 73–84. https://doi.org/10.1145/3401025.3401738

  30. Bschitter: benchmark-suite-serverless-computing (2021). https://github.com/Bschitter/benchmark-suite-serverless-computing Accessed 24 Dec 2021

  31. Eismann S, Costa DE, Liao L, Bezemer C-P, Shang W, van Hoorn A, Kounev S (2021) A case study on the stability of performance tests for serverless applications. arXiv:2107.13320 [cs]

  32. Tzenetopoulos A, Apostolakis E, Tzomaka A, Papakostopoulos C, Stavrakakis K, Katsaragakis M, Oroutzoglou I, Masouros D, Xydis S, Soudris D (2021) FaaS and curious, performance implications of serverless functions on edge computing platforms. In: Jagode H, Anzt H, Ltaief H, Luszczek P (eds) High performance computing. Lecture notes in computer science. Springer Cham, pp 428–438. https://doi.org/10.1007/978-3-030-90539-2_29

    Chapter  Google Scholar 

  33. Kjorveziroski V, Filiposka S, Trajkovik V (2021) Serverless platforms performance evaluation at the network edge. In: 13th ICT Innovations Conference 2021 Skopje, North Macedonia

  34. Wang I, Liri E, Ramakrishnan KK (2020) Supporting IoT applications with serverless edge clouds. In: 2020 IEEE 9th International Conference on Cloud Networking (CloudNet), pp 1–4.https://doi.org/10.1109/CloudNet51028.2020.9335805

  35. Agarwal S, Rodriguez MA, Buyya R (2021) A reinforcement learning approach to reduce serverless function cold start frequency. In: 2021 IEEE/ACM 21st International Symposium on Cluster Cloud and Internet Computing (CCGrid), pp 797–803. https://doi.org/10.1109/CCGrid51090.2021.00097

  36. Wang B, Ali-Eldin A, Shenoy P (2020) LaSS, running latency sensitive serverless computations at the edge. In: Proceedings of the 30th International Symposium on High-Performance Parallel and Distributed Computing, pp 239–251. Association for Computing Machinery New York. https://doi.org/10.1145/3431379.3460646

  37. Eiermann A, Renner M, Großmann M, Krieger UR (2017) On a fog computing platform built on ARM architectures by docker container technology. In: Eichler G, Erfurth C, Fahrnberger G (eds) Innovations for community services. Communications in computer and information science. Springer, Cham, pp 71–86. https://doi.org/10.1007/978-3-319-60447-3_6

    Chapter  Google Scholar 

  38. Kim J, Lee K (2019) FunctionBench, a suite of workloads for serverless cloud function service. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp 502–504. IEEE Milan, Italy. https://doi.org/10.1109/CLOUD.2019.00091

  39. kmu-bigdata/serverless-faas-workbench. BigData Lab. in KMU (2021). https://github.com/kmu-bigdata/serverless-faas-workbench Accessed 09 May 2021

  40. Aslanpour MS, Toosi AN, Cicconetti C, Javadi B, Sbarski P, Taibi D, Assuncao M, Gill SS, Gaire R, Dustdar S (2021) Serverless edge computing, vision and challenges. In: 2021 Australasian Computer Science Week Multiconference. ACSW ’21, pp 1–10. Association for Computing Machinery New York, NY, USA. https://doi.org/10.1145/3437378.3444367

  41. OpenFaaS—Serverless Functions Made Simple with Kubernetes. https://www.openfaas.com/ Accessed 26 April 2021

  42. Knative. https://knative.dev/ Accessed 10 April 2021

  43. Nuclio. https://nuclio.io/ Accessed 27 April 2021

  44. Kubeless. https://kubeless.io/ Accessed 26 April 2021

  45. openfaas/faas. OpenFaaS (2021). https://github.com/openfaas/faas Accessed 26 Dec 2021

  46. Autoscaling - OpenFaaS. https://docs.openfaas.com/architecture/autoscaling/ Accessed 26 Dec 2021

  47. Nguyen T-T, Yeom Y-J, Kim T, Park D-H, Kim S (2020) Horizontal Pod autoscaling in kubernetes for elastic container orchestration. Sensors 20(16):4621. https://doi.org/10.3390/s20164621

    Article  Google Scholar 

  48. Container Storage Interface (CSI) for Kubernetes GA (2019). https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/ Accessed 26 Dec 2021

  49. Kumar R, Trivedi MC (2021) Networking analysis and performance comparison of kubernetes CNI plugins. In: Bhatia SK, Tiwari S, Ruidan S, Trivedi MC, Mishra KK (eds) Advances in computer communication and computational sciences. Advances in intelligent systems and computing. Springer, Singapore, pp 99–109. https://doi.org/10.1007/978-981-15-4409-5_9

    Chapter  Google Scholar 

  50. Kubespray—Deploy a Production Ready Kubernetes Cluster. https://kubespray.io/#/ Accessed 26 Dec 2021

  51. Galal H, Introduction to K3s. https://www.suse.com/c/rancher_blog/introduction-to-k3s/ Accessed 18 Feb 2022

  52. K3s System Requirements. https://rancher.com/docs/k3s/latest/en/installation/installation-requirements/ Accessed 17 Feb 2022

  53. Kubeadm System Requirements. https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ Accessed 17 Feb 2022

  54. High availability (HA) | MicroK8s. http://microk8s.io Accessed 18 Feb 2022

  55. MicroK8s System Requirements. http://microk8s.io Accessed 17 Feb 2022

  56. MicroK8s 1.23 Release Notes. https://github.com/ubuntu/microk8s/releases Accessed 27 Dec 2021

  57. OpenFaaS Helm Chart for Kubernetes. https://github.com/openfaas/faas-netes Accessed 26 Dec 2021

  58. Dogan J (2021) rakyll/hey. https://github.com/rakyll/hey Accessed 26 Dec 2021

  59. Park J, Kim H, Lee K. (2020) Evaluating concurrent executions of multiple function-as-a-service runtimes with MicroVM. In: 2020 IEEE 13th International Conference on Cloud Computing (CLOUD), pp 532–536. IEEE Beijing, China. https://doi.org/10.1109/CLOUD49709.2020.00080. https://ieeexplore.ieee.org/document/9284320/ Accessed 17 Feb 2022

  60. Ustiugov D, Petrov P, Kogias M, Bugnion E, Grot B. (2021) Benchmarking, analysis, and optimization of serverless function snapshots. In: Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, pp 559–572. ACM Virtual USA. https://doi.org/10.1145/3445814.3446714. Accessed 17 Feb 2022

  61. Chadha M, Jindal A, Gerndt M (2021) Architecture-specific performance optimization of compute-intensive FaaS functions. arXiv: 2107.10008. Accessed 17 Feb 2022

  62. Choi J, Lee K (2020) Evaluation of network file system as a shared data storage in serverless computing. In: Proceedings of the 2020 Sixth International Workshop on Serverless Computing, pp 25–30. ACM Delft Netherlands. https://doi.org/10.1145/3429880.3430096. Accessed 17 Feb 2022

  63. Zhao L, Yang Y, Li Y, Zhou X, Li K (2021) Understanding, predicting and scheduling serverless workloads under partial interference. In: Proceedings of the International Conference for High Performance Computing Networking Storage and Analysis. SC ’21, pp 1–15. Association for Computing Machinery New York, NY, USA. https://doi.org/10.1145/3458817.3476215. Accessed 16 Feb 2022

  64. of-watchdog. OpenFaaS (2021). https://github.com/openfaas/of-watchdog Accessed 26 Dec 2021

  65. korvoj/k8s-distributions-iot-edge: Kubernetes distributions for the edge, serverless performance evaluation. https://github.com/korvoj/k8s-distributions-iot-edge Accessed 18 Feb 2022

  66. faasd - OpenFaaS. https://docs.openfaas.com/deployment/faasd/ Accessed 28 Feb 2022

  67. Patman J, Chemodanov D, Calyam P, Palaniappan K, Sterle C, Boccia M (2020) Predictive cyber foraging for visual cloud computing in large-scale IoT systems. IEEE Trans Netw Serv Manag 17(4):2380–2395. https://doi.org/10.1109/TNSM.2020.3010497

    Article  Google Scholar 

  68. Cho C, Shin S, Jeon H (2020) QoS-aware workload distribution in hierarchical edge clouds. A reinforcement learning approach. IEEE Access 8. https://doi.org/10.1109/ACCESS.2020.3033421

  69. Prometheus: Prometheus—Monitoring system & time series database. https://prometheus.io/ Accessed 26 Dec 2021

  70. Prometheus: Alertmanager | Prometheus. https://prometheus.io/docs/alerting/latest/alertmanager/ Accessed 26 Dec 2021

  71. Performance—OpenFaaS. https://docs.openfaas.com/architecture/performance/ Accessed 26 Dec 2021

  72. OpenFaaS Endpoint Load-Balancing. https://github.com/openfaas/faas-netes Accessed 26 Dec 2021

Download references

Acknowledgements

The work presented in this paper has received funding from the Faculty of Computer Science and Engineering under the “SCAP” project.

Author information

Author notes

  1. Vojdan Kjorveziroski and Sonja Filiposka equally contributed to this work.

Authors and Affiliations

  1. Faculty of Computer Science and Engineering, Ss. Cyril and Methodius University, Rudzer Boshkovikj 16, 1000, Skopje, North Macedonia

    Vojdan Kjorveziroski & Sonja Filiposka

Authors
  1. Vojdan Kjorveziroski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sonja Filiposka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vojdan Kjorveziroski.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kjorveziroski, V., Filiposka, S. Kubernetes distributions for the edge: serverless performance evaluation. J Supercomput 78, 13728–13755 (2022). https://doi.org/10.1007/s11227-022-04430-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04430-6

Keywords

Search

Navigation