skip to main content
10.1145/3313808.3313823acmconferencesArticle/Chapter ViewAbstractPublication PagesveeConference Proceedingsconference-collections
research-article

Tail latency in node.js: energy efficient turbo boosting for long latency requests in event-driven web services

Published: 14 April 2019 Publication History

Abstract

Cloud-based Web services are shifting to the event-driven, scripting language-based programming model to achieve productivity, flexibility, and scalability. Implementations of this model, however, generally suffer from long tail latencies, which we measure using Node.js as a case study. Unlike in traditional thread-based systems, reducing long tails is difficult in event-driven systems due to their inherent asynchronous programming model. We propose a framework to identify and optimize tail latency sources in scripted event-driven Web services. We introduce profiling that allows us to gain deep insights into not only how asynchronous event-driven execution impacts application tail latency but also how the managed runtime system overhead exacerbates the tail latency issue further. Using the profiling framework, we propose an event-driven execution runtime design that orchestrates the hardware’s boosting capabilities to reduce tail latency. We achieve higher tail latency reductions with lower energy overhead than prior techniques that are unaware of the underlying event-driven program execution model. The lessons we derive from Node.js apply to other event-driven services based on scripting language frameworks.

References

[1]
Wonsun Ahn, Jiho Choi, Thomas Shull, María J Garzarán, and Josep Torrellas. 2014. Improving JavaScript performance by deconstructing the type system. In Proc. of PLDI.
[2]
Mohammad Alizadeh, Abdul Kabbani, Tom Edsall, Balaji Prabhakar, Amin Vahdat, and Masato Yasuda. 2012. Less Is More: Trading a Little Bandwidth for Ultra-Low Latency in the Data Center. In Proc. of NSDI.
[3]
Luiz André Barroso, Jimmy Clidaras, and Urs Hölzle. 2013. The datacenter as a computer: An introduction to the design of warehouse-scale machines. Synthesis lectures on computer architecture 8, 3 (2013).
[4]
Sapan Bhatia, Charles Consel, and Julia Lawall. 2006. Memorymanager/Scheduler Co-design: Optimizing Event-driven Servers to Improve Cache Behavior. In Proc. of ISMM.
[5]
Edward A. Burton, Gerhard Schrom, Fabrice Paillet, Jonathan Douglas, William J. Lambert, Kaladhar Radhakrishnan, and Michael J. Hill. 2014. FIVR – Fully integrated voltage regulators on 4th generation Intel Core SoCs. In 2014 IEEE Applied Power Electronics Conference and Exposition - APEC 2014. 432–439.
[6]
Mike Cantelon, Marc Harter, TJ Holowaychuk, and Nathan Rajlich. 2013. Node.Js in Action. Manning Publications Co.
[7]
Md E. Haque, Yong hun Eom, Yuxiong He, Sameh Elnikety, Ricardo Bianchini, and Kathryn S. McKinley. 2015. Few-to-Many: Incremental Parallelism for Reducing Tail Latency in Interactive Services. In Proc. of ASPLOS.
[8]
Md E. Haque, Yuxiong He, Sameh Elnikety, Thu D. Nguyen, Ricardo Bianchini, and Kathryn McKinley. 2017. Exploiting Heterogeneity for Tail Latency and Energy Efficiency. In Proc. of MICRO.
[9]
Chun-Hung Hsiao, Cristiano L. Pereira, Jie Yu, Gilles A. Pokam, Satish Narayanasamy, Peter M. Chen, Ziyun Kong, and Jason Flinn. 2014. Race Detection for Event-Driven Mobile Applications. In Proc. of PLDI.
[10]
Chang-Hong Hsu, Yunqi Zhang, Michael A. Laurenzano, David Meisner, Thomas Wenisch, Lingjia Tang, Jason Mars, and Ronald G. Dreslinski. 2015. Adrenaline: Pinpointing and Reining in Tail Queries with Quick Voltage Boosting. In Proc. of HPCA.
[11]
James C. Hu, Irfan Pyarali, and Douglas C. Schmidt. 1997. High Performance Web Servers on Windows NT: Design and Performance. In Proc. of USENIX Windows NT Workshop.
[12]
The Innovative Computing Laboratory (ICL). 2016. PAPI 5.4.3 release. http://icl.cs.utk.edu/papi/software/view.html?id=245
[13]
Myeongjae Jeon, Yuxiong He, Hwanju Kim, Sameh Elnikety, Scott Rixner, and Alan L. Cox. 2016. TPC: Target-Driven Parallelism Combining Prediction and Correction to Reduce Tail Latency in Interactive Services. In Proc. of ASPLOS.
[14]
José A. Joao, Onur Mutlu, and Yale N. Patt. 2009. Flexible Referencecounting-based Hardware Acceleration for Garbage Collection. In Proc. of ISCA.
[15]
Richard Jones and Rafael D Lins. 1996. Garbage Collection: Algorithms for Automatic Dynamic Memory Management. Wiley.
[16]
Harshad Kasture, Davide B. Bartolini, Nathan Beckmann, and Daniel Sanchez. 2015. Rubik: Fast Analytical Power Management for Latencycritical Systems. In Proc. of MICRO.
[17]
Harshad Kasture and Daniel Sanchez. 2014. Ubik: Efficient Cache Sharing with Strict Qos for Latency-critical Workloads. In Proc. of ASPLOS.
[18]
Antoine Kaufmann, Simon Peter, Thomas Anderson, and Arvind Krishnamurthy. 2015. FlexNIC: Rethinking Network DMA. In Proc. of HotOS.
[19]
James R. Larus and Michael Parkes. 2002. Using Cohort Scheduling to Enhance Server Performance. In Proc. of USENIX ATC.
[20]
Jialin Li, Naveen Kr. Sharma, Dan R. K. Ports, and Steven D. Gribble. 2014. Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency. In Proc. of Cloud Computing.
[21]
David Lo, Liqun Cheng, Rama Govindaraju, Luiz AndrÃľ Barroso, and Christos Kozyrakis. 2014. Towards Energy Proportionality for Large-Scale Latency-Critical Workloads. In Proc. of ISCA.
[22]
Martin Maas, Krste Asanović, and John Kubiatowicz. 2018. A Hardware Accelerator for Tracing Garbage Collection. In Proc. of ISCA.
[23]
Martin Maas, Krste Asanović, Tim Harris, and John Kubiatowicz. 2016. Taurus: A Holistic Language Runtime System for Coordinating Distributed Managed-Language Applications. In Proc. of ASPLOS.
[24]
Magnus Madsen, Frank Tip, and Ondřej Lhoták. 2015. Static Analysis of Event-driven Node.Js JavaScript Applications. In Proc. of OOPSLA.
[25]
Takeshi Ogasawara. 2014. Workload characterization of server-side JavaScript. In Proc. of IISWC.
[26]
Vivek S. Pai, Peter Druschel, and Willy Zwaenepoel. 1999. Flash: An Efficient and Portable Web Server. In Proc. of USENIX ATC.
[27]
Boris Petrov, Martin Vechev, Manu Sridharan, and Julian Dolby. 2012. Race detection for web applications. In Proc. of PLDI.
[28]
Veselin Raychev, Martin Vechev, and Manu Sridharan. 2013. Effective race detection for event-driven programs. In Proc. of OOPSLA.
[29]
Jim Smith and Ravi Nair. 2005. Virtual machines: versatile platforms for systems and processes. Elsevier.
[30]
Gil Tene. 2018. wrk2: a HTTP benchmarking tool based mostly on wrk. https://github.com/giltene/wrk2
[31]
David Terei and Amit A. Levy. 2015. Blade: A Data Center Garbage Collector. CoRR (2015).
[32]
Matt Welsh, David Culler, and Eric Brewer. 2001. SEDA: an architecture for well-conditioned, scalable internet services. In Proc. of SOSP.
[33]
Matt Welsh, Steven D Gribble, Eric A Brewer, and David Culler. 2000. A design framework for highly concurrent systems. In TR UCB/CSD-00-1108.
[34]
Xi Yang, Stephen M. Blackburn, and Kathryn S. McKinley. 2016. Elfen Scheduling: Fine-grain Principled Borrowing from Latency-critical Workloads Using Simultaneous Multithreading. In Proc. of USENIX.
[35]
Yunqi Zhang, David Meisner, Jason Mars, and Lingjia Tang. 2016. Treadmill: Attributing the Source of Tail Latency through Precise Load Testing and Statistical Inference. In Proc. of ISCA.
[36]
Yuhao Zhu, Matthew Halpern, and Vijay Janapa Reddi. 2015. EventBased Scheduling for Energy-Efficient QoS (eQoS) in Mobile Web Applications. In Proc. of HPCA.
[37]
Yuhao Zhu, Daniel Richins, Wenzhi Cui, Matthew Halpern, and Vijay Janapa Reddi. 2016. Node Benchmarks. https://github.com/ nodebenchmark/benchmarks
[38]
Yuhao Zhu, Daniel Richins, Matthew Halpern, and Vijay Janapa Reddi. 2015. Microarchitectural implications of event-driven server-side web applications. In Proc. of MICRO.

Cited By

View all
  • (2023)A Study of Java Microbenchmark Tail LatenciesCompanion of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578245.3584690(77-81)Online publication date: 15-Apr-2023
  • (2022)Supervisory Event Loop-based Autoscaling of Node.js Deployments2022 International Conference on High Performance Big Data and Intelligent Systems (HDIS)10.1109/HDIS56859.2022.9991325(1-7)Online publication date: 10-Dec-2022
  • (2022)Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunitiesFrontiers of Computer Science10.1007/s11704-020-0072-316:6Online publication date: 22-Jan-2022

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
VEE 2019: Proceedings of the 15th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments
April 2019
206 pages
ISBN:9781450360203
DOI:10.1145/3313808
  • General Chair:
  • Jennifer Sartor,
  • Program Chairs:
  • Mayur Naik,
  • Chris Rossbach
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 April 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. JavaScript
  2. Node.js
  3. event-driven
  4. tail latency

Qualifiers

  • Research-article

Conference

VEE '19

Acceptance Rates

Overall Acceptance Rate 80 of 235 submissions, 34%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)12
  • Downloads (Last 6 weeks)2
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)A Study of Java Microbenchmark Tail LatenciesCompanion of the 2023 ACM/SPEC International Conference on Performance Engineering10.1145/3578245.3584690(77-81)Online publication date: 15-Apr-2023
  • (2022)Supervisory Event Loop-based Autoscaling of Node.js Deployments2022 International Conference on High Performance Big Data and Intelligent Systems (HDIS)10.1109/HDIS56859.2022.9991325(1-7)Online publication date: 10-Dec-2022
  • (2022)Performance optimization for cloud computing systems in the microservice era: state-of-the-art and research opportunitiesFrontiers of Computer Science10.1007/s11704-020-0072-316:6Online publication date: 22-Jan-2022

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media