skip to main content
10.1145/3343737.3343750acmconferencesArticle/Chapter ViewAbstractPublication PagesapsysConference Proceedingsconference-collections
research-article

USETL: Unikernels for Serverless Extract Transform and Load Why should you settle for less?

Published: 19 August 2019 Publication History

Abstract

Growing popularity of serverless functions is driving the need to optimize the execution platform to reduce resource usage and increase the number of functions that can be executed concurrently. This reduces the provider's costs and increases profit. While current serverless solutions use containers and/or virtual machines, we propose a unikernel based design called USETL which is specialized for serverless extract, transform, load (ETL) workloads. Our design is motivated by a number of key observations: serverless functions are stateless, ephemeral and event-driven. Further, each function's specific purpose is known at invocation time. Unikernels are a natural fit for execution contexts with these properties: they are minimal kernels packaged with a single application in a single address space, which makes them incredibly lightweight. Our design removes network and storage components entirely, replacing them with high level APIs tailored to the needs of serverless ETL functions. Virtualizing I/O at the runtime library interface reduces memory and CPU overheads, yielding higher consolidation density.

References

[1]
{n.d.}. 1631: gVisor runsc guest->host breakout via filesystem cache desync. https://bugs.chromium.org/p/project-zero/issues/detail?id=1631. Accessed: 2019-03-25.
[2]
{n.d.}. 43. Vhost Library - Dataplane Development Kit. https://doc.dpdk.org/guides-18.08/prog_guide/vhost_lib.html Accessed: 2019-03-01.
[3]
{n.d.}. abelg/virtual_io_acceleration: Virtual I/O acceleration technologies for KVM. https://github.com/abelg/virtual_io_acceleration. Accessed: 2019-03-10.
[4]
{n.d.}. AWS Lambda - Serverless Compute. https://aws.amazon.com/lambda/. Accessed: 2019-03-10.
[5]
{n.d.}. Azure Functions - Serverless Architecture. https://azure.microsoft.com/en-us/services/functions/. Accessed: 2019-03-10.
[6]
{n.d.}. Cloud Functions - Event-driven Serverless Computing. https://cloud.google.com/functions/. Accessed: 2019-03-10.
[7]
{n.d.}. Cloud Functions - Overview. https://www.ibm.com/cloud/functions. Accessed: 2019-03-10.
[8]
{n.d.}. Configure SR-IOV Network Virtual Functions in Linux KVM. https://software.intel.com/en-us/articles/configure-sr-iov- network-virtual-functions-in-linux-kvm. Accessed: 2019-04-24.
[9]
{n.d.}. CVE security vulnerability database. https://www.cvedetails.com/product/47/Linux-Linux-Kernel.html?vendor_id=33. Accessed: 2019-03-05.
[10]
{n.d.}. Firecracker - Lightweight Virtualization for Serverless Computing. https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/. Accessed: 2019-01-28.
[11]
{n.d.}. google/gvisor: Container Runtime Sandbox. https://github.com/google/gvisor. Accessed: 2019-03-25.
[12]
{n.d.}. kvm: the Linux Virtual Machine Monitor. https://www.kernel.org/doc/ols/2007/ols2007v1-pages-225-230.pdf. Accessed: 2019-05-03.
[13]
{n.d.}. Measuring the Horizontal Attack Profile of Nabla Containers. https://blog.hansenpartnership.com/measuring-the-horizontal-attack-profile-of-nabla-containers/. Accessed: 2019-03-25.
[14]
{n.d.}. Networking --- KVM. https://www.linux-kvm.org/page/Networking. Accessed: 2019-07-15.
[15]
{n.d.}. OSv - the operating system designed for the cloud. http://osv.io/. Accessed: 2019-02-27.
[16]
{n.d.}. Privilege Escalation in gVisor, Google's Container Sandbox. https://justi.cz/security/2018/11/14/gvisor-lpe.html. Accessed: 2019-03-25.
[17]
{n.d.}. Repo rumpkernel/wiki Wiki. https://github.com/rumpkernel/wiki/wiki/Repo. Accessed: 2019-03-05.
[18]
{n.d.}. Rump Kernels. http://rumpkernel.org/. Accessed: 2019-03-05.
[19]
{n.d.}. rumpkernel/rumprun: The Rumprun unikernel and toolchain for various platforms. https://github.com/rumpkernel/rumprun. Accessed: 2019-03-05.
[20]
{n.d.}. Serverless by the number: 2018 report. https://serverless.com/blog/serverless-by-the-numbers-2018-data-report/. Accessed: 2019-02-27.
[21]
{n.d.}. State of the Cloud Report. https://www.rightscale.com/lp/state-of-the-cloud. Accessed: 2019-01-28.
[22]
{n.d.}. UKL: A Unikernel Based on Linux - now + next. https://next.redhat.com/2018/11/14/ukl-a-unikernel-based-on-linux/. Accessed: 2019-02-27.
[23]
{n.d.}. vhost_net: a kernel-level virtio server. https://lwn.net/Articles/346267/. Accessed: 2019-04-24.
[24]
Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-Performance Serverless Computing. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 923--935. https://www.usenix.org/conference/atc18/presentation/akkus
[25]
Lixiang Ao, Liz Izhikevich, Geoffrey M. Voelker, and George Porter. 2018. Sprocket: A Serverless Video Processing Framework. In Proceedings of the ACM Symposium on Cloud Computing, SoCC 2018, Carlsbad, CA, USA, October 11-13, 2018. 263--274.
[26]
Sean Barker, Timothy Wood, Prashant Shenoy, and Ramesh Sitaraman. 2012. An Empirical Study of Memory Sharing in Virtual Machines. In Presented as part of the 2012 USENIX Annual Technical Conference (USENIX ATC 12). USENIX, Boston, MA, 273--284. https://www.usenix.org/conference/atc12/technical-sessions/presentation/barker
[27]
D. R. Engler, M. F. Kaashoek, and J. O'Toole, Jr. 1995. Exokernel: An Operating System Architecture for Application-level Resource Management. In Proceedings of the Fifteenth ACM Symposium on Operating Systems Principles (SOSP '95). ACM, New York, NY, USA, 251--266.
[28]
Sadjad Fouladi, Riad S. Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. 2017. Encoding, Fast and Slow: Low-Latency Video Processing Using Thousands of Tiny Threads. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 363--376. https://www.usenix.org/conference/nsdi17/technical-sessions/presentation/fouladi
[29]
Antti Kantee and Justin Cormack. 2014. Rump Kernels: No OS? No Problem! ;login: 39, 5 (2014). https://www.usenix.org/publications/login/october-2014-vol-39-no-5/rump-kernels-no-os-no-problem
[30]
Sanidhya Kashyap, Changwoo Min, and Taesoo Kim. 2018. Scaling Guest OS Critical Sections with eCS. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 159--172. https://www.usenix.org/conference/atc18/presentation/kashyap
[31]
Ralph Kimball, Laura Reeves, Warren Thornthwaite, Margy Ross, and Warren Thornwaite. 1998. The Data Warehouse Lifecycle Toolkit: Expert Methods for Designing, Developing and Deploying Data Warehouses with CD Rom (1st ed.). John Wiley & Sons, Inc., New York, NY, USA.
[32]
Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis. 2018. Pocket: Elastic Ephemeral Storage for Serverless Analytics. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 427--444. https://www.usenix.org/conference/osdi18/presentation/klimovic
[33]
Ricardo Koller and Dan Williams. 2017. Will Serverless End the Dominance of Linux in the Cloud?. In Proceedings of the 16th Workshop on Hot Topics in Operating Systems (HotOS '17). ACM, New York, NY, USA, 169--173.
[34]
Yossi Kuperman, Eyal Moscovici, Joel Nider, Razya Ladelsky, Abel Gordon, and Dan Tsafrir. 2016. Paravirtual Remote I/O. In Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '16). ACM, New York, NY, USA, 49--65.
[35]
Wenhao Li, Yubin Xia, Haibo Chen, Binyu Zang, and Haibing Guan. 2015. Reducing World Switches in Virtualized Environment with Flexible Cross-world Calls. In Proceedings of the 42Nd Annual International Symposium on Computer Architecture (ISCA '15). ACM, New York, NY, USA, 375--387.
[36]
James Litton, Anjo Vahldiek-Oberwagner, Eslam Elnikety, Deepak Garg, Bobby Bhattacharjee, and Peter Druschel. 2016. Light-Weight Contexts: An OS Abstraction for Safety and Performance. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16). USENIX Association, Savannah, GA, 49--64. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/litton
[37]
Anil Madhavapeddy, Thomas Leonard, Magnus Skjegstad, Thomas Gazagnaire, David Sheets, Dave Scott, Richard Mortier, Amir Chaudhry, Balraj Singh, Jon Ludlam, Jon Crowcroft, and Ian Leslie. 2015. Jitsu: Just-In-Time Summoning of Unikernels. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). USENIX Association, Oakland, CA, 559--573. https://www.usenix.org/conference/nsdi15/technical-sessions/presentation/madhavapeddy
[38]
Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: Library Operating Systems for the Cloud. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '13). ACM, New York, NY, USA, 461--472.
[39]
Filipe Manco, Costin Lupu, Florian Schmidt, Jose Mendes, Simon Kuenzer, Sumit Sati, Kenichi Yasukata, Costin Raiciu, and Felipe Huici. 2017. My VM is Lighter (and Safer) Than Your Container. In Proceedings of the 26th Symposium on Operating Systems Principles (SOSP '17). ACM, New York, NY, USA, 218--233.
[40]
Joao Martins, Mohamed Ahmed, Costin Raiciu, and Felipe Huici. 2013. Enabling Fast, Dynamic Network Processing with clickOS. In Proceedings of the Second ACM SIGCOMM Workshop on Hot Topics in Software Defined Networking (HotSDN '13). ACM, New York, NY, USA, 67--72.
[41]
Avantika Mathur, Mingming Cao, Suparna Bhattacharya, Andreas Dilger, Alex Tomas, and Laurent Vivier. 2007. The new ext4 filesystem: current status and future plans. In Proceedings of the Linux symposium, Vol. 2. 21--33.
[42]
Edward Oakes, Leon Yang, Dennis Zhou, Kevin Houck, Tyler Harter, Andrea Arpaci-Dusseau, and Remzi Arpaci-Dusseau. 2018. SOCK: Rapid Task Provisioning with Serverless-Optimized Containers. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 57--70. https://www.usenix.org/conference/atc18/presentation/oakes
[43]
Diego Ongaro, Alan L. Cox, and Scott Rixner. 2008. Scheduling I/O in Virtual Machine Monitors. In Proceedings of the Fourth ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '08). ACM, New York, NY, USA, 1--10.
[44]
Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2014. Arrakis: The Operating System is the Control Plane. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14). USENIX Association, Broomfield, CO, 1--16. https://www.usenix.org/conference/osdi14/technical-sessions/presentation/peter
[45]
Qifan Pu, Shivaram Venkataraman, and Ion Stoica. 2019. Shuffling, Fast and Slow: Scalable Analytics on Serverless Infrastructure. In 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). USENIX Association, Boston, MA, 193--206. https://www.usenix.org/conference/nsdi19/presentation/pu
[46]
Xiang Song, Jicheng Shi, Haibo Chen, and Binyu Zang. 2013. Schedule Processes, Not VCPUs. In Proceedings of the 4th Asia-Pacific Workshop on Systems (APSys '13). ACM, New York, NY, USA, Article 1, 7 pages.
[47]
Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. 2018. Peeking Behind the Curtains of Serverless Platforms. In 2018 USENIX Annual Technical Conference (USENIX ATC 18). USENIX Association, Boston, MA, 133--146. https://www.usenix.org/conference/atc18/presentation/wang-liang
[48]
Chuliang Weng, Zhigang Wang, Minglu Li, and Xinda Lu. 2009. The Hybrid Scheduling Framework for Virtual Machine Systems. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '09). ACM, New York, NY, USA, 111--120.

Cited By

View all
  • (2024)Function as a Service (FaaS) for Fast, Efficient, Scalable SystemsServerless Computing Concepts, Technology and Architecture10.4018/979-8-3693-1682-5.ch008(134-151)Online publication date: 5-Apr-2024
  • (2024)A Lightweight File System Design for UnikernelApplied Sciences10.3390/app1408334214:8(3342)Online publication date: 16-Apr-2024
  • (2024)SURE: Secure Unikernels Make Serverless Computing Rapid and EfficientProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698558(668-688)Online publication date: 20-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
APSys '19: Proceedings of the 10th ACM SIGOPS Asia-Pacific Workshop on Systems
August 2019
115 pages
ISBN:9781450368933
DOI:10.1145/3343737
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 August 2019

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

APSys '19
Sponsor:

Acceptance Rates

APSys '19 Paper Acceptance Rate 15 of 36 submissions, 42%;
Overall Acceptance Rate 169 of 430 submissions, 39%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)44
  • Downloads (Last 6 weeks)5
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Function as a Service (FaaS) for Fast, Efficient, Scalable SystemsServerless Computing Concepts, Technology and Architecture10.4018/979-8-3693-1682-5.ch008(134-151)Online publication date: 5-Apr-2024
  • (2024)A Lightweight File System Design for UnikernelApplied Sciences10.3390/app1408334214:8(3342)Online publication date: 16-Apr-2024
  • (2024)SURE: Secure Unikernels Make Serverless Computing Rapid and EfficientProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698558(668-688)Online publication date: 20-Nov-2024
  • (2024)uIO: Lightweight and Extensible UnikernelsProceedings of the 2024 ACM Symposium on Cloud Computing10.1145/3698038.3698518(580-599)Online publication date: 20-Nov-2024
  • (2024)Sandboxing Functions for Efficient and Secure Multi-tenant Serverless DeploymentsProceedings of the 2nd Workshop on SErverless Systems, Applications and MEthodologies10.1145/3642977.3652096(25-31)Online publication date: 22-Apr-2024
  • (2024)Pronghorn: Effective Checkpoint Orchestration for Serverless Hot-StartsProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3629556(298-316)Online publication date: 22-Apr-2024
  • (2024)FUSIONIZE++: Improving Serverless Application Performance Using Dynamic Task Inlining and Infrastructure OptimizationIEEE Transactions on Cloud Computing10.1109/TCC.2024.345110812:4(1172-1185)Online publication date: Oct-2024
  • (2024)Meunik: Rethinking Virtual Machine Memory Resource Management for Unikernel-Based VMs2024 IEEE/ACM Symposium on Edge Computing (SEC)10.1109/SEC62691.2024.00030(292-305)Online publication date: 4-Dec-2024
  • (2024)A Survey of Unikernel Security: Insights and Trends from a Quantitative Analysis2024 Cyber Awareness and Research Symposium (CARS)10.1109/CARS61786.2024.10778787(1-9)Online publication date: 28-Oct-2024
  • (2024)Prebaking runtime environments to improve the FaaS cold start latencyFuture Generation Computer Systems10.1016/j.future.2024.01.019155:C(287-299)Online publication date: 1-Jun-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media