Carlos Segarra
Peter Pietzuch
ABSTRACT
Serverless computing allows users to execute pieces of code (so called functions) on-demand in the cloud without having to provision any hardware resources. However, by executing in the cloud and delegating control over hardware resources, the integrity of the execution and the confidentiality of function code and data are at the mercy of the cloud provider and serverless runtime. Confidential computing aims to remove trust from the cloud provider by executing applications inside hardware enclaves. In spite of the increasing adoption of confidential computing, designing a confidential serverless runtime with moderate performance overhead remains an open challenge.
In this short article we present our experience porting the Knative serverless runtime to a confidential setting using Confidential Containers (CoCo), a technology that allows the execution of unmodified (encrypted) container images inside confidential VMs (cVMs). Our results show that cVMs are not ready to execute container-based serverless functions. Starting a serverless function in a CoCo from an encrypted container image with attestation takes up to 17 seconds. Starting 16 serverless functions concurrently takes more than three minutes, 20× slower than its non-confidential counterpart. We analyze the main sources of overhead, and outline the research challenges to bridge the gap between confidential and serverless computing.
- 2021. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC.Google Scholar
- Mania Abdi, Samuel Ginzburg, Xiayue Charles Lin, Jose Faleiro, Gohar Irfan Chaudhry, Inigo Goiri, Ricardo Bianchini, Daniel S Berger, and Rodrigo Fonseca. 2023. Palette Load Balancing: Locality Hints for Serverless Functions. In Proceedings of the Eighteenth European Conference on Computer Systems (Rome, Italy) (EuroSys '23). Association for Computing Machinery, New York, NY, USA, 365--380. Google ScholarDigital Library
- Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 419--434. https://www.usenix.org/conference/nsdi20/presentation/agacheGoogle Scholar
- Ayaz Akram, Anna Giannakou, Venkatesh Akella, Jason Lowe-Power, and Sean Peisert. 2021. Performance Analysis of Scientific Computing Workloads on General Purpose TEEs. 1066--1076. Google ScholarCross Ref
- Fritz Alder, N Asokan, Arseny Kurnikov, Andrew Paverd, and Michael Steiner. 2019. S-faas: Trustworthy and accountable function-as-a-service using intel SGX. In Proceedings of the 2019 ACM SIGSAC Conference on Cloud Computing Security Workshop.Google ScholarDigital Library
- Mohamed Alzayat, Jonathan Mace, Peter Druschel, and Deepak Garg. 2023. Groundhog: Efficient Request Isolation in FaaS. In Proceedings of the Eighteenth European Conference on Computer Systems (Rome, Italy) (EuroSys '23). Association for Computing Machinery, New York, NY, USA, 398--415. Google ScholarDigital Library
- AMD. 2022. AMD Secure Encrypted Virtualization. https://developer.amd.com/sev/.Google Scholar
- AMD. 2023. Confidential Computing Performance - Google Cloud C2D VM Instances. https://www.amd.com/system/files/documents/3rd-gen-epyc-gcp-c2d-conf-compute-perf-brief.pdf.Google Scholar
- AMD. 2023. Microsoft Azure Confidential Computing Powered by 3rd Gen EPYC CPUs. https://community.amd.com/t5/epyc-processors/microsoft-azure-confidential-computing-powered-by-3rd-gen-epyc/ba-p/497796.Google Scholar
- AntStack. 2024. Serverless For Unstructured Data Problems in Life Sciences. https://www.antstack.com/blog/how-serverless-is-solving-unstructured-data-problem-for-life-sciences/.Google Scholar
- archlinux Wiki. 2024. init. https://wiki.archlinux.org/title/init.Google Scholar
- Arm. 2022. Arm TrustZone. https://www.arm.com/technologies/trustzone-for-cortex-a.Google Scholar
- Aws. 2024. Real-time fraud detection using AWS serverless and machine learning services. https://aws.amazon.com/blogs/machine-learning/real-time-fraud-detection-using-aws-serverless-and-machine-learning-services/.Google Scholar
- Microsoft Azure. 2024. Confidential Containers on Azure Container Instances. https://learn.microsoft.com/en-us/azure/container-instances/container-instances-confidential-overview.Google Scholar
- Maurice Bailleu, Dimitra Giantsidi, Vasilis Gavrielatos, Do Le Quoc, Vijay Nagarajan, and Pramod Bhatotia. 2021. Avocado: A Secure In-Memory Distributed Storage System. In 2021 USENIX Annual Technical Conference (USENIX ATC 21). USENIX Association, 65--79. https://www.usenix.org/conference/atc21/presentation/bailleuGoogle Scholar
- James Bottomley. 2024. QEMU Mailing List - sev: enable secret injection to a self described area in OVMF. https://lore.kernel.org/qemu-devel/[email protected]/.Google Scholar
- Stefan Brenner and Rüdiger Kapitza. 2019. Trust more, serverless. In Proceedings of the 12th ACM International Conference on Systems and Storage.Google ScholarDigital Library
- Marc Brooker, Mike Danilov, Chris Greenwood, and Phil Piwonka. 2023. On-demand Container Loading in AWS Lambda. In 2023 USENIX Annual Technical Conference (USENIX ATC 23). USENIX Association, Boston, MA, 315--328. https://www.usenix.org/conference/atc23/presentation/brookerGoogle Scholar
- James Cadden, Thomas Unger, Yara Awad, Han Dong, Orran Krieger, and Jonathan Appavoo. 2020. SEUSS: Skip Redundant Paths to Make Serverless Fast. In Proceedings of the Fifteenth European Conference on Computer Systems (Heraklion, Greece) (EuroSys '20). Association for Computing Machinery, New York, NY, USA, Article 32, 15 pages. Google ScholarDigital Library
- Google Cloud. 2022. Confidential Computing. https://cloud.google.com/confidential-computing.Google Scholar
- Google Cloud. 2022. Ubiquitous Data Encryption. https://cloud.google.com/compute/confidential-vm/docs/ubiquitous-data-encryption.Google Scholar
- Google Cloud. 2024. What is a Virtual Machine? https://cloud.google.com/learn/what-is-a-virtual-machine.Google Scholar
- Confidential Computing Consortium. 2022. Confidential Computing - Open Source Community. https://confidentialcomputing.io/.Google Scholar
- containerd. 2024. An industry-standard container runtime with an emphasis on simplicity, robustness and portability. https://containerd.io/.Google Scholar
- containerd. 2024. Runtime v2. https://github.com/containerd/containerd/tree/main/runtime/v2.Google Scholar
- Containers. 2024. OCIcrypt - Encryption libraries for OCI container images. https://github.com/containers/ocicrypt.Google Scholar
- Containers. 2024. Skopeo - Work with remote image registries. https://github.com/containers/skopeo.Google Scholar
- Confidential Containers. 2024. Attestation Agent. https://github.com/confidential-containers/guest-components/tree/main/attestation-agent.Google Scholar
- Confidential Containers. 2024. Confidential Containers - Overview. https://github.com/confidential-containers/confidential-containers/blob/main/overview.md.Google Scholar
- Confidential Containers. 2024. Generic Key Broker Service. https://github.com/confidential-containers/kbs.Google Scholar
- Confidential Containers. 2024. image-rs - Container Images Rust Crate. https://github.com/confidential-containers/guest-components/tree/main/image-rs.Google Scholar
- Confidential Containers. 2024. Key Broker Client. https://github.com/confidential-containers/guest-components/tree/main/attestation-agent/kbc.Google Scholar
- Confidential Containers. 2024. Welcome to Confidential Containers! https://confidentialcontainers.org/.Google Scholar
- Kata Containers. 2023. The speed of containers, the security of VMs. https://katacontainers.io/.Google Scholar
- Kata Containers. 2024. Kata Agent. https://github.com/kata-containers/kata-containers/blob/main/src/agent/README.md.Google Scholar
- Kata Containers. 2024. Kata Agent API - Github. https://github.com/kata-containers/kata-containers/blob/CCv0/src/runtime/virtcontainers/kata_agent.go_L2518-L2531.Google Scholar
- Kata Containers. 2024. Kata Containers Architecture. https://github.com/kata-containers/kata-containers/tree/main/docs/design/architecture.Google Scholar
- Kata Containers. 2024. Kata Open Policy Agent. https://github.com/kata-containers/kata-containers/tree/main/src/kata-opa.Google Scholar
- Open Containers. 2023. runc - CLI tool for spawning and running containers according to the OCI specification. https://github.com/opencontainers/runc.Google Scholar
- Open Containers. 2024. OCI Image Format Specification. https://github.com/opencontainers/image-spec.Google Scholar
- DockerHub. 2024. registry - Distribution implementation for storing and distributing container images and artifacts. https://hub.docker.com/_/registry.Google Scholar
- Knative Serving Docs. 2023. Hello World - Python. https://github.com/knative/docs/tree/main/code-samples/serving/hello-world/helloworld-python.Google Scholar
- enclave cc. 2024. Process-based Confidential Container Runtime. https://github.com/confidential-containers/enclave-cc.Google Scholar
- Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 363--376. https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/fouladiGoogle ScholarDigital Library
- Anders Tungeland Gjerdrum, Håvard Dagenborg Johansen, Lars Brenna, and Dag Johansen. 2019. Diggi: A Secure Framework for Hosting Native Cloud Functions with Minimal Trust. In 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA). 18--27. Google ScholarCross Ref
- Gramine. 2024. Gramine Project - a library OS for Unmodified Applications. https://gramineproject.io/.Google Scholar
- Brendan Gregg. 2023. Flame Graphs. https://www.brendangregg.com/flamegraphs.html.Google Scholar
- Red Hat. 2024. Attestation in Confidential Computing. https://www.redhat.com/en/blog/attestation-confidential-computing.Google Scholar
- Red Hat. 2024. Confidential computing use cases. https://www.redhat.com/en/blog/confidential-computing-use-cases.Google Scholar
- Red Hat. 2024. Understanding the Confidential Containers Attestation Flow. https://www.redhat.com/en/blog/understanding-confidential-containers-attestation-flow.Google Scholar
- IBM. 2023. IBM Cloud. https://www.ibm.com/cloud.Google Scholar
- IBM. 2023. IBM Cloud Bare Metal Servers. https://www.ibm.com/products/bare-metal-servers.Google Scholar
- Apache Incubator. 2021. Teaclave. https://github.com/apache/incubator-teaclave.Google Scholar
- Intel. 2022. Intel Software Guard Extensions. https://www.intel.co.uk/content/www/uk/en/architecture-and-technology/software-guard-extensions.html.Google Scholar
- Intel. 2024. Intel TDX - CCC Linux Guest Hardening. https://intel.github.io/ccc-linux-guest-hardening-docs/security-spec.html.Google Scholar
- Intel. 2024. Intel Trust Domain Extensions. https://www.intel.com/content/www/us/en/developer/tools/trust-domain-extensions/overview.html.Google Scholar
- Vatche Ishakian, Vinod Muthusamy, and Aleksander Slominski. 2018. Serving Deep Learning Models in a Serverless Platform. In IEEE International Conference on Cloud Engineering, (IC2E).Google ScholarCross Ref
- Zhipeng Jia and Emmett Witchel. 2021. Boki: Stateful Serverless Computing with Shared Logs. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles (Virtual Event, Germany) (SOSP '21). Association for Computing Machinery, New York, NY, USA, 691--707. Google ScholarDigital Library
- Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the Cloud: Distributed Computing for the 99%. In ACM Symposium on Cloud Computing (SOCC).Google ScholarDigital Library
- Artjom Joosen, Ahmed Hassan, Martin Asenov, Rajkarn Singh, Luke Darlow, Jianfeng Wang, and Adam Barker. 2023. How Does It Function? Characterizing Long-Term Trends in Production Serverless Workloads. In Proceedings of the 2023 ACM Symposium on Cloud Computing (, Santa Cruz, CA, USA,) (SoCC '23). Association for Computing Machinery, New York, NY, USA, 443--458. Google ScholarDigital Library
- David Kaplan. 2016. AMD x86 Memory Encryption Technologies. USENIX Association, Austin, TX.Google Scholar
- David Kaplan. 2023. Hardware VM Isolation in the Cloud: Enabling confidential computing with AMD SEV-SNP technology. Queue 21, 4 (sep 2023), 49--67. Google ScholarDigital Library
- Knative. 2024. Knative is an Open-Source Enterprise-level solution to build Serverless and Event Driven Applications. https://knative.dev/docs/.Google Scholar
- Knative. 2024. Knative Serving Architecture. https://knative.dev/docs/serving/architecture/.Google Scholar
- Knative. 2024. Tag Resolution. https://knative.dev/docs/serving/tag-resolution/.Google Scholar
- Kubernetes. 2024. CRI - Container Runtime Interface. https://kubernetes.io/docs/concepts/architecture/cri/.Google Scholar
- Kubernetes. 2024. kubelet. https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/.Google Scholar
- Linux KVM. 2024. Kernel Virtual Machine. https://linux-kvm.org/page/Main_Page.Google Scholar
- Ashraf Mahgoub, Edgardo Barsallo Yi, Karthick Shankar, Sameh Elnikety, Somali Chaterji, and Saurabh Bagchi. 2022. ORION and the Three Rights: Sizing, Bundling, and Prewarming for Serverless DAGs. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). USENIX Association, Carlsbad, CA, 303--320. https://www.usenix.org/conference/osdi22/presentation/mahgoubGoogle Scholar
- Linux manual page. 2024. namespaces. https://man7.org/linux/man-pages/man7/namespaces.7.html.Google Scholar
- Microsoft. 2020. Microsoft Azure Attestation. https://docs.microsoft.com/azure/attestation/overview.Google Scholar
- Microsoft. 2022. Microsoft Azure Confidential Computing. https://azure.microsoft.com/en-gb/solutions/confidential-compute/.Google Scholar
- Microsoft. 2023. Inside Look: How Azure Linux powers Confidential Containers on AKS. https://techcommunity.microsoft.com/t5/linux-and-open-source-blog/inside-look-how-azure-linux-powers-confidential-containers-on/ba-p/3981296.Google Scholar
- Microsoft. 2024. Azure Functions - Execute event-driven serverless code with an end-to-end development experience. https://azure.microsoft.com/en-us/products/functions/.Google Scholar
- Nydus. 2024. Nydus - Acceleration Framework For Container Image. https://nydus.dev/.Google Scholar
- QEMU Options. 2023. RAM. https://wiki.gentoo.org/wiki/QEMU/Options_RAM.Google Scholar
- OVMF. 2024. AMD SEV x64 Package. https://github.com/tianocore/edk2/blob/master/OvmfPkg/AmdSev/AmdSevX64.dsc.Google Scholar
- The Washington Post. 2024. NSA infiltrates links to Yahoo, Google data centers worldwide, Snowden documents say. https://www.washingtonpost.com/world/national-security/nsa-infiltrates-links-to-yahoo-google-data-centers-worldwide-snowden-documents-say/2013/10/30/e51d661e-4166-11e3-8b74-d89d714ca4dd_story.html.Google Scholar
- Qemu. 2024. Qemu - A generic and open-source machine emulator and virtualizer. https://www.qemu.org/.Google Scholar
- Qemu. 2024. QEMU Firmware Configuration Device. https://www.qemu.org/docs/master/specs/fw_cfg.html.Google Scholar
- Quay. 2024. Quay Container Registry. https://quay.io/.Google Scholar
- Github Container Registry. 2024. Your packages, at home with their code. https://github.com/features/packages.Google Scholar
- IBM Research. 2024. LPC 2021 - Attestation and Secret Injection for Confidential VMs, Containers, and Pods. https://lpc.events/event/11/contributions/994/.Google Scholar
- Alireza Sahraei, Soteris Demetriou, Amirali Sobhgol, Haoran Zhang, Abhigna Nagaraja, Neeraj Pathak, Girish Joshi, Carla Souza, Bo Huang, Wyatt Cook, Andrii Golovei, Pradeep Venkat, Andrew Mcfague, Dimitrios Skarlatos, Vipul Patel, Ravinder Thind, Ernesto Gonzalez, Yun Jin, and Chunqiang Tang. 2023. XFaaS: Hyperscale and Low Cost Serverless Functions at Meta. 231--246. Google ScholarDigital Library
- SeaBIOS. 2023. SeaBIOS. https://www.seabios.org/SeaBIOS.Google Scholar
- Kaspersky Security. 2024. Downgrade Attack. https://encyclopedia.kaspersky.com/glossary/downgrade-attack/.Google Scholar
- Amazon Web Services. 2024. AWS Lambda - Run code without thinking of servers or clusters. https://aws.amazon.com/lambda/.Google Scholar
- Knative Serving. 2024. Configuring Scale to Zero. https://knative.dev/docs/serving/autoscaling/scale-to-zero/.Google Scholar
- Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Batum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 205--218. https://www.usenix.org/conference/atc20/presentation/shahradGoogle Scholar
- Simon Shillaker and Peter Pietzuch. 2020. Faasm: Lightweight Isolation for Efficient Stateful Serverless Computing. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 419--433. https://www.usenix.org/conference/atc20/presentation/shillakerGoogle Scholar
- Sigstore. 2024. Cosign - Container Signing. https://github.com/sigstore/cosign.Google Scholar
- Brijesh Singh. 2024. [PATCH v9 00/43] Add AMD Secure Nested Paging (SEV-SNP) Guest Support. https://lore.kernel.org/linux-mm/[email protected]/t/.Google Scholar
- UEFI Platform Initialization Specification. 2023. Driver Execution Environment (DXE) Phase. https://uefi.org/specs/PI/1.8/V2_Overview.html.Google Scholar
- Edgless Systems. 2024. The world's most secure Kubernetes. https://www.edgeless.systems/products/constellation/.Google Scholar
- Tianocore. 2024. OVMF - Open Virtual Machine Firmware. https://github.com/tianocore/tianocore.github.io/wiki/OVMF.Google Scholar
- Bohdan Trach, Oleksii Oleksenko, Franz Gregor, Pramod Bhatotia, and Christof Fetzer. 2019. Clemmys: Towards secure remote execution in FaaS. In Proceedings of the 12th ACM International Conference on Systems and Storage.Google ScholarDigital Library
- VirTEE. 2024. Calculate AMD SEV/SEV-ES/SEV-SNP measurement for confidential computing. https://github.com/virtee/sev-snp-measure.Google Scholar
- VMWare. 2024. Introduction to vSockets. https://vdc-repo.vmware.com/vmwb-repository/dcr-public/a49be05e-fa6d-4da1-9186-922fbfef149e/a65f3c51-aaeb-476d-80c3-827b805c2f9e/doc/vsockAbout.3.2.html.Google Scholar
- Jinpeng Wei and Calton Pu. 2005. TOCTTOU Vulnerabilities in UNIX-Style File Systems: An Anatomical Study. In 4th USENIX Conference on File and Storage Technologies (FAST 05). USENIX Association, San Francisco, CA. https://www.usenix.org/conference/fast-05/tocttou-vulnerabilities-unix-style-file-systems-anatomical-studyGoogle Scholar
- Xingda Wei, Fangming Lu, Tianxia Wang, Jinyu Gu, Yuhan Yang, Rong Chen, and Haibo Chen. 2023. No Provisioned Concurrency: Fast RDMA-codesigned Remote Fork for Serverless Computing. In 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23). USENIX Association, Boston, MA, 497--517. https://www.usenix.org/conference/osdi23/presentation/wei-rdmaGoogle Scholar
- AMD Whitepaper. 2024. AMD SEV-SNP: Strengthening VM Isolation with Integrity Protection and More. https://www.amd.com/content/dam/amd/en/documents/epyc-business-docs/white-papers/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf.Google Scholar
- Wenting Zheng, Ankur Dave, Jethro G. Beekman, Raluca Ada Popa, Joseph E. Gonzalez, and Ion Stoica. 2017. Opaque: An Oblivious and Encrypted Distributed Analytics Platform. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 283--298. https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/zhengGoogle ScholarDigital Library
Index Terms
- Serverless Confidential Containers: Challenges and Opportunities
Index terms have been assigned to the content through auto-classification.
Recommendations
Towards Seamless Serverless Computing Across an Edge-Cloud Continuum
UCC '23: Proceedings of the IEEE/ACM 16th International Conference on Utility and Cloud ComputingServerless computing has emerged as an attractive paradigm due to the efficiency of development and the ease of deployment without managing any underlying infrastructure. Nevertheless, serverless computing approaches face numerous challenges to unlock ...
The SPEC cloud group's research vision on FaaS and serverless architectures
WoSC '17: Proceedings of the 2nd International Workshop on Serverless ComputingCloud computing enables an entire ecosystem of developing, composing, and providing IT services. An emerging class of cloud-based software architectures, serverless, focuses on providing software architects the ability to execute arbitrary functions ...
Practical Tooling for Serverless Computing
UCC '17: Proceedings of the10th International Conference on Utility and Cloud ComputingCloud applications are increasingly built from a mixture of runtime technologies. Hosted functions and service-oriented web hooks are among the most recent ones which are natively supported by cloud platforms. They are collectively referred to as ...
Comments