skip to main content
10.1145/3491084.3491426acmconferencesArticle/Chapter ViewAbstractPublication PagesmiddlewareConference Proceedingsconference-collections
research-article

The serverless shell

Published: 06 December 2021 Publication History

Abstract

Serverless computing is a recent paradigm to program the cloud. It allows to execute user-defined functions at scale, on demand, and in a pay-per-use manner. This paper reports on adapting the Unix shell for serverless. Our software, called the serverless shell (sshell), runs shell scripts on a serverless platform much like with a regular computer. It permits to reuse an existing code base while benefiting from the massive power of serverless and paying only for the resources used. sshell is built around a small set of components that includes a new inter-process communication layer for serverless. We evaluate it in AWS Lambda using several microbenchmarks and a large-scale application. Our results show that sshell achieves comparable or better performance than a high-end server. Moreover, it can be faster and more cost-efficient than a cluster-based solution to mine large datasets.

References

[1]
Alexandra Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). USENIX Association, Santa Clara, CA, 419--434.
[2]
Amazon. 2014. AWS Lambda Layer. https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html.
[3]
Amazon. 2016. AWS Step Functions. https://aws.amazon.com/step-functions.
[4]
Daniel Barcelona-Pons, Marc Sánchez-Artigas, Gerard París, Pierre Sutra, and Pedro García-López. 2019. On the FaaS Track: Building Stateful Distributed Applications with Serverless Architectures. In Proceedings of the 20th International Middleware Conference (Davis, CA, USA) (Middleware '19). ACM, New York, NY, USA, 41--54.
[5]
Daniel J. Barrett, Richard E. Silverman, and Robert G. Byrnes. 2005. SSH, the Secure Shell: The Definitive Guide. O'Reilly Media, Inc.
[6]
Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, and John Wilkes. 2016. Borg, Omega, and Kubernetes. Queue 14, 1, Article 10 (Jan. 2016), 24 pages.
[7]
Brent Chun and Andrew McNabb. 2009. https://code.google.com/archive/p/parallel-ssh
[8]
John Cristy. 1990. ImageMagick. https://imagemagick.org/
[9]
Santo Fortunato, Marián Boguñá, Alessandro Flammini, and Filippo Menczer. 2008. Approximating PageRank from In-Degree. In Algorithms and Models for the Web-Graph, William Aiello, Andrei Broder, Jeannette Janssen, and Evangelos Milios (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 59--71.
[10]
Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, Shuvo Chatterjee, Christos Kozyrakis, Matei Zaharia, and Keith Winstein. 2019. From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). USENIX Association, Renton, WA, 475--488.
[11]
Eric Hammond. 2014. lambdash. https://github.com/alestic/lambdash
[12]
IOpipe. 2016. lambda-shell. https://github.com/iopipe/lambda-shell
[13]
Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. 2017. Occupy the Cloud: Distributed Computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing (SoCC'17).
[14]
Aurèle Mahéo and Pierre Sutra. 2021. The Serverless Shell. https://github.com/crucial-project/serverless-shell
[15]
Francesco Marchioni and Manik Surtani. 2012. Infinispan Data Grid Platform. Packt Publishing Ltd.
[16]
Elliott I. Organick. 1972. The Multics System: An Examination of Its Structure. MIT Press, Cambridge, MA, USA.
[17]
Google Cloud Platform. 2018. Cloud Composer. https://cloud.google.com/composer.
[18]
Remko Popma. 2017. Picocli. https://picocli.info
[19]
Deepti Raghavan, Sadjad Fouladi, Philip Levis, and Matei Zaharia. 2020. POSH: A Data-Aware Shell. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 617--631.
[20]
Dennis M. Ritchie and Ken Thompson. 1974. The UNIX Time-Sharing System. Commun. ACM 17, 7 (July 1974), 365--375.
[21]
Amazon Web Services. 2021. Amazon EFS quotas and limits. https://docs.aws.amazon.com/efs/latest/ug/limits.html#limits-fs-specific
[22]
Sergey Shnitkind. 2019. Linkrun. https://github.com/trendsci/linkrun/
[23]
Vikram Sreekanti, Chenggang Wu, Xiayue Charles Lin, Johann Schleier-Smith, Jose M. Faleiro, Joseph E. Gonzalez, Joseph M. Hellerstein, and Alexey Tumanov. 2020. Cloudburst: Stateful Functions-as-a-Service. arXiv:cs.DC/2001.04592
[24]
Ole Tange. 2011. GNU Parallel: The Command-Line Power Tool. login Usenix Mag. 36, 1 (2011).
[25]
Nikos Vasilakis, Konstantinos Kallas, Konstantinos Mamouras, Achilles Benetopoulos, and Lazar Cvetković. 2021. PaSh: Light-Touch Data-Parallel Shell Processing. In Proceedings of the Sixteenth European Conference on Computer Systems (Online Event, United Kingdom) (EuroSys '21). Association for Computing Machinery, New York, NY, USA, 49--66.
[26]
Andy B. Yoo, Morris A. Jette, and Mark Grondona. 2003. SLURM: Simple Linux Utility for Resource Management. In Job Scheduling Strategies for Parallel Processing, Dror Feitelson, Larry Rudolph, and Uwe Schwiegelshohn (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 44--60.
[27]
Haoran Zhang, Adney Cardoza, Peter Baile Chen, Sebastian Angel, and Vincent Liu. 2020. Fault-tolerant and transactional stateful serverless workflows. In 14th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2020, Virtual Event, November 4-6, 2020. USENIX Association, 1187--1204.

Cited By

View all
  • (2024)sAirflow: Adopting Serverless in a Legacy Workflow SchedulerEuro-Par 2024: Parallel Processing10.1007/978-3-031-69577-3_18(254-268)Online publication date: 26-Aug-2024
  • (2023)Executing Shell Scripts in the Wrong Order, CorrectlyProceedings of the 19th Workshop on Hot Topics in Operating Systems10.1145/3593856.3595891(103-109)Online publication date: 22-Jun-2023
  • (2022)CrossFit: Fine-grained Benchmarking of Serverless Application Performance across Cloud Providers2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC56403.2022.00016(51-60)Online publication date: Dec-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
Middleware '21: Proceedings of the 22nd International Middleware Conference: Industrial Track
December 2021
35 pages
ISBN:9781450391528
DOI:10.1145/3491084
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

  • USENIX Assoc: USENIX Assoc
  • IFIP

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 December 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. function-as-a-service
  2. serverless computing
  3. shell

Qualifiers

  • Research-article

Funding Sources

  • EU Horizon

Conference

Middleware '21
Sponsor:
Middleware '21: 22nd International Middleware Conference
December 6 - 10, 2021
Québec city, Canada

Acceptance Rates

Overall Acceptance Rate 203 of 948 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)44
  • Downloads (Last 6 weeks)6
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)sAirflow: Adopting Serverless in a Legacy Workflow SchedulerEuro-Par 2024: Parallel Processing10.1007/978-3-031-69577-3_18(254-268)Online publication date: 26-Aug-2024
  • (2023)Executing Shell Scripts in the Wrong Order, CorrectlyProceedings of the 19th Workshop on Hot Topics in Operating Systems10.1145/3593856.3595891(103-109)Online publication date: 22-Jun-2023
  • (2022)CrossFit: Fine-grained Benchmarking of Serverless Application Performance across Cloud Providers2022 IEEE/ACM 15th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC56403.2022.00016(51-60)Online publication date: Dec-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media