skip to main content
10.1145/3102980.3103003acmconferencesArticle/Chapter ViewAbstractPublication PageshotosConference Proceedingsconference-collections
research-article
Open access

Return of the Runtimes: Rethinking the Language Runtime System for the Cloud 3.0 Era

Published: 07 May 2017 Publication History

Abstract

The public cloud is moving to a Platform-as-a-Service model where services such as data management, machine learning or image classification are provided by the cloud operator while applications are written in high-level languages and leverage these services.
Managed languages such as Java, Python or Scala are widely used in this setting. However, while these languages can increase productivity, they are often associated with problems such as unpredictable garbage collection pauses or warm-up overheads.
We argue that the reason for these problems is that current language runtime systems were not initially designed for the cloud setting. To address this, we propose seven tenets for designing future language runtime systems for cloud data centers. We then outline the design of a general substrate for building such runtime systems, based on these seven tenets.

References

[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: A System for Large-Scale Machine Learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI '16).
[2]
Apache Arrow. 2017. Powering Columnar In-Memory Analytics. (2017). https://arrow.apache.org/
[3]
Krste Asanovic and D Patterson. 2014. Firebox: A hardware building block for 2020 warehouse-scale computers. In USENIX FAST, Vol. 13.
[4]
Microsoft Azure. 2017. Machine Learning. (2017). https://azure.microsoft.com/en-us/services/machine-learning/
[5]
Andrew Baumann, Paul Barham, Pierre-Evariste Dagand, Tim Harris, Rebecca Isaacs, Simon Peter, Timothy Roscoe, Adrian Schüpbach, and Akhilesh Singhania. 2009. The Multikernel: A New OS Architecture for Scalable Multicore Systems. In 22nd Symposium on Operating Systems Principles (SOSP '09).
[6]
Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. 2014. IX: A Protected Dataplane Operating System for High Throughput and Low Latency. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI '14).
[7]
Stephen M. Blackburn, Perry Cheng, and Kathryn S. McKinley. 2004. Oil and Water? High Performance Garbage Collection in Java with MMTk. In 26th International Conference on Software Engineering (ICSE '04).
[8]
Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, and John Wilkes. 2016. Borg, Omega, and Kubernetes. Commun. ACM 59, 5 (April 2016), 50--57.
[9]
Adrian M Caulfield, Eric S Chung, Andrew Putnam, Hari Angepat, Jeremy Fowers, Michael Haselman, Stephen Heil, Matt Humphrey, Puneet Kaur, Joo-Young Kim, et al. 2016. A Cloud-Scale Acceleration Architecture. In 49th Annual IEEE/ACM International Symposium on Microarchitecture.
[10]
Huawei Press Center. 2017. Huawei proposed DC 3.0 architecture of future data center to meet the requirement of real-time data processing in big data era. (2017). http://pr.huawei.com/en/news/hw-423134-3.0.htm
[11]
Yunji Chen, Tianshi Chen, Zhiwei Xu, Ninghui Sun, and Olivier Temam. 2016. DianNao Family: Energy-efficient Hardware Accelerators for Machine Learning. Commun. ACM 59, 11 (Oct. 2016), 105--112.
[12]
Eric S. Chung, John D. Davis, and Jaewon Lee. 2013. LINQits: Big Data on Little Clients. In 40th International Symposium on Computer Architecture (ISCA '13).
[13]
Databricks. 2015. Project Tungsten: Bringing Spark Closer to Bare Metal. (2015). https://databricks.com/blog/2015/04/28/project-tungsten-bringing-spark-closer-to-bare-metal.html
[14]
Jesper de Jong. 2015. Project Valhalla -- Value Types. (2015). http://www.jesperdj.com/2015/10/04/project-valhalla-value-types/
[15]
Paolo Faraboschi, Kimberly Keeton, Tim Marsland, and Dejan Milojicic. 2015. Beyond Processor-centric Operating Systems. In 15th Workshop on Hot Topics in Operating Systems (HotOS XV).
[16]
Peter X. Gao, Akshay Narayan, Sagar Karandikar, Joao Carreira, Sangjin Han, Rachit Agarwal, Sylvia Ratnasamy, and Scott Shenker. Network Requirements for Resource Disaggregation. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI '16).
[17]
Nicolas Geoffray, Gaël Thomas, Julia Lawall, Gilles Muller, and Bertil Folliot. VMKit: A Substrate for Managed Runtime Environments. In 6th ACM International Conference on Virtual Execution Environments (VEE '10).
[18]
Ionel Gog, Jana Giceva, Malte Schwarzkopf, Kapil Viswani, Dimitrios Vytiniotis, Ganesan Ramalingan, Manuel Costa, Derek Murray, Steven Hand, and Michael Isard. 2015. Broom: sweeping out Garbage Collection from Big Data systems. In 15th Workshop on Hot Topics in Operating Systems (HotOS XV).
[19]
Ionel Gog, Malte Schwarzkopf, Natacha Crooks, Matthew P. Grosvenor, Allen Clement, and Steven Hand. 2015. Musketeer: All for One, One for All in Data Processing Systems. In EuroSys '15.
[20]
Google. 2017. Google App Engine: Platform as a Service. (2017). https://developers.google.com/appengine
[21]
Shan Shan Huang, Amir Hormati, David F. Bacon, and Rodric Rabbah. 2008. Liquid Metal: Object-Oriented Programming Across the Hardware/Software Boundary. In ECOOP '08.
[22]
InfoQ. 2017. Azul Systems Launches Falcon, a New Just-in-Time Compiler for Java, Based on LLVM. (2017). https://www.infoq.com/news/2017/05/azul-falcon
[23]
Intel. 2017. Intel® Rack Scale Design. (2017). http://www.intel.com/content/www/us/en/architecture-and-technology/rack-scale-design-overview.html
[24]
Mick Jordan, Laurent Daynès, Grzegorz Czajkowski, Marcin Jarzab, and Ciarán Bryce. 2004. Scaling J2EE Application Servers with the Multi-tasking Virtual Machine. Technical Report. Sun Microsystems, Inc., Mountain View, CA, USA.
[25]
Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson, et al. 2017, arXiv:1704.04760. In-Datacenter Performance Analysis of a Tensor Processing Unit. (2017, arXiv:1704.04760).
[26]
Data Center Knowledge. 2013. Meet the Future of Data Center Rack Technologies. (Feb. 2013). http://www.datacenterknowledge.com/archives/2013/02/20/meet-the-future-of-data-center-rack-technologies/
[27]
David Koeplinger, Christina Delimitrou, Raghu Prabhakar, Christos Kozyrakis, Yaqi Zhang, and Kunle Olukotun. 2016. Automatic Generation of Efficient Accelerators for Reconfigurable Hardware. In 43rd International Symposium on Computer Architecture (ISCA '16).
[28]
Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization (CGO '04).
[29]
David Lion, Adrian Chiu, Hailong Sun, Xin Zhuang, Nikola Grcevski, and Ding Yuan. 2016. Don't Get Caught in the Cold, Warm-up Your JVM: Understand and Eliminate JVM Warm-up Overhead in Data-Parallel Systems. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI '16).
[30]
Martin Maas, Krste Asanovic, and John Kubiatowicz. Grail Quest: A New Proposal for Hardware-assisted Garbage Collection. In 6th Workshop on Architectures and Systems for Big Data (ASBD '16).
[31]
Martin Maas, Krste Asanović, Tim Harris, and John Kubiatowicz. 2016. Taurus: A Holistic Language Runtime System for Coordinating Distributed Managed-Language Applications. In 21st International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '16).
[32]
Martin Maas, Tim Harris, Krste Asanovic, and John Kubiatowicz. Trash Day: Coordinating Garbage Collection in Distributed Systems. In 5th Workshop on Hot Topics in Operating Systems (HotOS XV).
[33]
Anil Madhavapeddy, Richard Mortier, Charalampos Rotsos, David Scott, Balraj Singh, Thomas Gazagnaire, Steven Smith, Steven Hand, and Jon Crowcroft. 2013. Unikernels: Library Operating Systems for the Cloud. In 18th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '13).
[34]
Microsoft Developer Network. 2017. Application Domains. (2017). https://msdn.microsoft.com/en-us/library/2bh4z9hs(v=vs.110).aspx
[35]
Microsoft Developer Network. 2017. Ngen.exe (Native Image Generator). (2017). https://msdn.microsoft.com/en-us/library/6t9t5wcf(v=vs.110).aspx
[36]
Khanh Nguyen, Lu Fang, Guoqing Xu, Brian Demsky, Shan Lu, Sanazsadat Alamian, and Onur Mutlu. 2016. Yak: A High-Performance Big-Data-Friendly Garbage Collector. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI '16).
[37]
Khanh Nguyen, Kai Wang, Yingyi Bu, Lu Fang, Jianfei Hu, and Guoqing Xu. 2015. FACADE: A Compiler and Runtime for (Almost) Object-Bounded Big Data Applications. In 20th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '15).
[38]
OpenJDK. 2017. JEP 295: Ahead-of-Time Compilation. (2017). http://openjdk.java.net/jeps/295
[39]
Kay Ousterhout, Ryan Rasti, Sylvia Ratnasamy, Scott Shenker, and Byung-Gon Chun. 2015. Making Sense of Performance in Data Analytics Frameworks. In 12th Symposium on Networked Systems Design and Implementation (NSDI '15).
[40]
Jian Ouyang, Shiding Lin, Wei Qi, Yong Wang, Bo Yu, Song Jiang, undefined, undefined, undefined, and undefined. 2014. SDA: Software-defined accelerator for large-scale DNN systems. 2014 IEEE Hot Chips 26 Symposium (2014).
[41]
Shoumik Palkar, James J Thomas, Anil Shanbhag, Deepak Narayanan, Holger Pirk, Malte Schwarzkopf, Saman Amarasinghe, Matei Zaharia, and Stanford InfoLab. Weld: A Common Runtime for High Performance Data Analytics. In 8th biennial Conference on Innovative Data Systems Research (CIDR '17).
[42]
Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2014. Arrakis: The Operating System is the Control Plane. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI '14).
[43]
Google Cloud Platform. 2017. Cloud Dataflow - Batch & Stream Data Processing. (2017). https://cloud.google.com/dataflow/
[44]
Tom's IT Pro. 2016. Cloud 3.0 And 'Building Scale' At Interop. (May 2016). http://www.tomsitpro.com/articles/interop-cloud-3-building-scale,1-3277.html
[45]
Android Open Source Project. 2017. Implementing ART Just-In-Time (JIT) Compiler. (2017). https://source.android.com/devices/tech/dalvik/jit-compiler.html
[46]
Christopher J. Rossbach, Yuan Yu, Jon Currey, Jean-Philippe Martin, and Dennis Fetterly. 2013. Dandelion: A Compiler and Runtime for Heterogeneous Systems. In 24th ACM Symposium on Operating Systems Principles (SOSP '13).
[47]
Amazon Web Services. 2017. Amazon EC2 F1 Instances. (2017). http://aws.amazon.com/ec2/instance-types/f1/
[48]
Amazon Web Services. 2017. Amazon Machine Learning - Predictive Analytics with AWS. (2017). http://aws.amazon.com/machine-learning/
[49]
Amazon Web Services. 2017. AWS Lambda - Serverless Compute. (2017). http://aws.amazon.com/lambda/
[50]
Gil Tene, Balaji Iyengar, and Michael Wolf. 2011. C4: The Continuously Concurrent Compacting Collector. In ISMM '11.
[51]
Christian Wimmer and Thomas Würthinger. 2012. Truffle: A Self-optimizing Runtime System. In 3rd Annual Conference on Systems, Programming, and Applications: Software for Humanity (SPLASH '12).

Cited By

View all
  • (2022)FireworksProceedings of the Seventeenth European Conference on Computer Systems10.1145/3492321.3519581(663-677)Online publication date: 28-Mar-2022
  • (2021)Generating High-Performance FPGA Accelerator Designs for Big Data Analytics with Fletcher and Apache ArrowJournal of Signal Processing Systems10.1007/s11265-021-01650-6Online publication date: 1-Mar-2021
  • (2020)Open-source Serverless Architectures: an Evaluation of Apache OpenWhisk2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC48980.2020.00052(329-335)Online publication date: Dec-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HotOS '17: Proceedings of the 16th Workshop on Hot Topics in Operating Systems
May 2017
185 pages
ISBN:9781450350686
DOI:10.1145/3102980
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Cloud 3.0
  2. Data Centers
  3. FPGAs
  4. Garbage Collection
  5. JIT Compilation
  6. Managed Language Runtime Systems
  7. Platform-as-a-Service
  8. Resource Disaggregation
  9. Serverless Computing

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

HotOS '17
Sponsor:
HotOS '17: Workshop on Hot Topics in Operating Systems
May 7 - 10, 2017
BC, Whistler, Canada

Upcoming Conference

HOTOS '25
Workshop on Hot Topics in Operating Systems
May 14 - 16, 2025
Banff , AB , Canada

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)147
  • Downloads (Last 6 weeks)17
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)FireworksProceedings of the Seventeenth European Conference on Computer Systems10.1145/3492321.3519581(663-677)Online publication date: 28-Mar-2022
  • (2021)Generating High-Performance FPGA Accelerator Designs for Big Data Analytics with Fletcher and Apache ArrowJournal of Signal Processing Systems10.1007/s11265-021-01650-6Online publication date: 1-Mar-2021
  • (2020)Open-source Serverless Architectures: an Evaluation of Apache OpenWhisk2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC)10.1109/UCC48980.2020.00052(329-335)Online publication date: Dec-2020
  • (2019)Rebooting Virtualization Research (Again)Proceedings of the 10th ACM SIGOPS Asia-Pacific Workshop on Systems10.1145/3343737.3343746(99-106)Online publication date: 19-Aug-2019
  • (2019)Fletcher: A Framework to Efficiently Integrate FPGA Accelerators with Apache Arrow2019 29th International Conference on Field Programmable Logic and Applications (FPL)10.1109/FPL.2019.00051(270-277)Online publication date: Sep-2019
  • (2019) GC-WiseFuture Generation Computer Systems10.1016/j.future.2019.05.027100:C(674-688)Online publication date: 1-Nov-2019
  • (2018)Latency and throughput characterization of convolutional neural networks for mobile computer visionProceedings of the 9th ACM Multimedia Systems Conference10.1145/3204949.3204975(204-215)Online publication date: 12-Jun-2018
  • (2018)Evaluation of Production Serverless Computing Environments2018 IEEE 11th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD.2018.00062(442-450)Online publication date: Jul-2018
  • (2018)Big Data in Cloud Computing: A Review of Key Technologies and Open IssuesAdvances in Internet, Data & Web Technologies10.1007/978-3-319-75928-9_45(504-513)Online publication date: 24-Feb-2018

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media