research-article

jVerbs: ultra-low latency for data center applications

Authors:

Patrick Stuedi,

Bernard Metzler,

Animesh TrivediAuthors Info & Claims

SOCC '13: Proceedings of the 4th annual Symposium on Cloud Computing

Article No.: 10, Pages 1 - 14

https://doi.org/10.1145/2523616.2523631

Published: 01 October 2013 Publication History

Abstract

Network latency has become increasingly important for data center applications. Accordingly, several efforts at both hardware and software level have been made to reduce the latency in data centers. Limited attention, however, has been paid to network latencies of distributed systems running inside an application container such as the Java Virtual Machine (JVM) or the .NET runtime.

In this paper, we first highlight the latency overheads observed in several well-known Java-based distributed systems. We then present jVerbs, a networking framework for the JVM which achieves bare-metal latencies in the order of single digit microseconds using methods of Remote Direct Memory Access (RDMA). With jVerbs, applications are mapping the network device directly into the JVM, cutting through both the application virtual machine and the operating system. In the paper, we discuss the design and implementation of jVerbs and demonstrate how it can be used to improve latencies in some of the popular distributed systems running in data centers.

References

[1]

Apache HBase with MemStore-Local Allocation Buffers. http://blog.cloudera.com/blog/.

[2]

Distributed Query Execution Engine using Apache HDFS. https://github.com/cloudera/impala.

[3]

Netty: Asynchronous Event-Driven Network Application Framework. http://netty.io.

[4]

Off-heap Cache for the Java Virtual Machine. http://directmemory.apache.org.

[5]

Socket Direct Protocol http://www.infinibandta.org/specs.

[6]

Softiwarp http://www.gitorious.org/softiwarp.

[7]

Spymemcached http://code.google.com/p/spymemcached.

[8]

M. Alizadeh, A. Kabbani, T. Edsall, B. Prabhakar, A. Vahdat, and M. Yasuda. Less is More: Trading a Little Bandwidth for Ultra-low Latency in the Data Center. In Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, NSDI'12, pages 19--19, Berkeley, CA, USA, 2012. USENIX Association.

Digital Library

[9]

A. Bilas and E. W. Felten. Fast RPC on the SHRIMP Virtual Memory Mapped Network Interface. J. Parallel Distrib. Comput., 40(1): 138--146, Jan. 1997.

Digital Library

[10]

A. D. Birrell and B. J. Nelson. Implementing Remote Procedure Calls. ACM Trans. Comput. Syst., 2(1): 39--59, Feb. 1984.

Digital Library

[11]

M. A. Blumrich, K. Li, R. Alpert, C. Dubnicki, E. W. Felten, and J. Sandberg. Virtual Memory Mapped Network Interface for the SHRIMP Multicomputer. SIGARCH Comput. Archit. News, 22(2): 142--153, Apr. 1994.

Digital Library

[12]

D. Borthakur, J. Gray, J. S. Sarma, K. Muthukkaruppan, N. Spiegelberg, H. Kuang, K. Ranganathan, D. Molkov, A. Menon, S. Rash, R. Schmidt, and A. Aiyer. Apache Hadoop Goes Realtime at Facebook. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data, SIGMOD '11, pages 1071--1080, New York, NY, USA, 2011. ACM.

Digital Library

[13]

P. Buonadonna, A. Geweke, and D. Culler. An Implementation and Analysis of the Virtual Interface Architecture. In Proceedings of the 1998 ACM/IEEE conference on Supercomputing (CDROM), Supercomputing '98, pages 1--15, Washington, DC, USA, 1998. IEEE Computer Society.

Digital Library

[14]

C. Chang and T. von Eicken. A Software Architecture for Zero-Copy RPC in Java. Technical report, Ithaca, NY, USA, 1998.

Digital Library

[15]

C.-C. Chang and T. von Eicken. Interfacing Java to the Virtual Interface Architecture. In Proceedings of the ACM 1999 conference on Java Grande, JAVA '99, pages 51--57, New York, NY, USA, 1999. ACM.

Digital Library

[16]

C. Engle, A. Lupher, R. Xin, M. Zaharia, M. J. Franklin, S. Shenker, and I. Stoica. Shark: Fast Data Dnalysis using Coarse-grained Distributed Memory. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, SIGMOD '12, pages 689--692, New York, NY, USA, 2012. ACM.

Digital Library

[17]

P. Frey. Zero-Copy Network Communication: An Applicability Study of iWARP beyond Micro Benchmarks, Dissertation. http://e-collection.library.ethz.ch/view/eth:1653.

[18]

S. Hefty. Rsockets, 2012 OpenFabris International Workshop, Monterey, CA, USA. 2012.

[19]

J. Hilland, P. Culley, J. Pinkerton, and R. Recio. RDMA Protocol Verbs Specification (Version 1.0). http://www.rdmaconsortium.org/home/draft-hilland-iwarp-verbs-v1.0-RDMAC.pdf.

[20]

J. Jose, H. Subramoni, M. Luo, M. Zhang, J. Huang, M. Wasi-ur Rahman, N. S. Islam, X. Ouyang, H. Wang, S. Sur, and D. K. Panda. Memcached Design on High Performance RDMA Capable Interconnects. In Proceedings of the 2011 International Conference on Parallel Processing, ICPP '11, pages 743--752, Washington, DC, USA, 2011. IEEE Computer Society.

Digital Library

[21]

S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton, and T. Vassilakis. Dremel: Interactive Analysis of Web-Scale Datasets. Proc. VLDB Endow., 3(1--2): 330--339, Sept. 2010.

Digital Library

[22]

C. Mitchell, Y. Geng, and J. Li. Using One-Sided RDMA Reads to Build a Fast, CPU-Efficient Key-Value Store. In Proceedings of the 2013 USENIX Annual Technical Conference, USENIX ATC'13, Berkeley, CA, USA, 2013. USENIX Association.

Digital Library

[23]

J. Ousterhout, P. Agrawal, D. Erickson, C. Kozyrakis, J. Leverich, D. Mazières, S. Mitra, A. Narayanan, D. Ongaro, G. Parulkar, M. Rosenblum, S. M. Rumble, E. Stratmann, and R. Stutsman. The Case for RAMCloud. Commun. ACM, 54(7): 121--130, July 2011.

Digital Library

[24]

P. Stuedi, A. Trivedi, and B. Metzler. Wimpy Nodes with 10GbE: Leveraging One-sided Operations in Soft-RDMA to Boost Memcached. In Proceedings of the 2012 USENIX Annual Technical Conference, USENIX ATC'12, pages 31--31, Berkeley, CA, USA, 2012. USENIX Association.

Digital Library

[25]

A. Trivedi, B. Metzler, and P. Stuedi. A Case for RDMA in Clouds: Turning Supercomputer Networking into Commodity. In Proceedings of the Second Asia-Pacific Workshop on Systems, APSys '11, pages 17:1--17:5, New York, NY, USA, 2011. ACM.

Digital Library

[26]

B. C. Vattikonda, G. Porter, A. Vahdat, and A. C. Snoeren. Practical TDMA for Datacenter Ethernet. In Proceedings of the 7th ACM European Conference on Computer Systems, EuroSys '12, pages 225--238, New York, NY, USA, 2012. ACM.

Digital Library

[27]

T. von Eicken, A. Basu, V. Buch, and W. Vogels. U-Net: a User-level Network Interface for Parallel and Distributed Computing. In Proceedings of the fifteenth ACM symposium on Operating systems principles, SOSP '95, pages 40--53, New York, NY, USA, 1995. ACM.

Digital Library

[28]

M. Welsh and D. Culler. Jaguar: Enabling Efficient Communication and I/O in Java. In Concurrency: Practice and Experience, Special Issue on Java for High-Performance Applications, 1999.

Cited By

Ruhland FKrakowski FSchöttner M(2023)Transparent network acceleration for big data computing in Java2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00339(2406-2413)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TrustCom60117.2023.00339
Eslahi-Kelorazi MLe LPedone F(2023)Heron: Scalable State Machine Replication on Shared Memory2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58367.2023.00025(138-150)Online publication date: Jun-2023
https://doi.org/10.1109/DSN58367.2023.00025
Smit M(2022)Ping times: Relating economic growth to internet connectivityCanadian Journal of Regional Science10.7202/1092249ar45:2(99-106)Online publication date: 13-Sep-2022
https://doi.org/10.7202/1092249ar
Show More Cited By

Index Terms

jVerbs: ultra-low latency for data center applications
1. Networks
  1. Network protocols
2. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles

Recommendations

SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices
VEE '16

Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned ...
SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices
VEE '16: Proceedings of the12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments

Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned ...
Nosv

nOSV can provide a bare-metal like performance for HPC applications on Cloud.The CPU cores and main memory are not shared among guest VMs of nOSV.Dedicated I/O resources are allocated to I/O sensitive HPC guests.Other virtualization environments can run ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SOCC '13: Proceedings of the 4th annual Symposium on Cloud Computing

October 2013

427 pages

ISBN:9781450324281

DOI:10.1145/2523616

General Chair:
Guy Lohman

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Conference

SOCC '13

Sponsor:

SOCC '13: ACM Symposium on Cloud Computing

October 1 - 3, 2013

California, Santa Clara

Acceptance Rates

SOCC '13 Paper Acceptance Rate 23 of 114 submissions, 20%;

Overall Acceptance Rate 169 of 722 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
644
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ruhland FKrakowski FSchöttner M(2023)Transparent network acceleration for big data computing in Java2023 IEEE 22nd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom)10.1109/TrustCom60117.2023.00339(2406-2413)Online publication date: 1-Nov-2023
https://doi.org/10.1109/TrustCom60117.2023.00339
Eslahi-Kelorazi MLe LPedone F(2023)Heron: Scalable State Machine Replication on Shared Memory2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN58367.2023.00025(138-150)Online publication date: Jun-2023
https://doi.org/10.1109/DSN58367.2023.00025
Smit M(2022)Ping times: Relating economic growth to internet connectivityCanadian Journal of Regional Science10.7202/1092249ar45:2(99-106)Online publication date: 13-Sep-2022
https://doi.org/10.7202/1092249ar
Taranov KByan SMarathe VHoefler TIves ZBonifati AEl Abbadi A(2022)KafkaDirect: Zero-copy Data Access for Apache Kafka over RDMA NetworksProceedings of the 2022 International Conference on Management of Data10.1145/3514221.3526056(2191-2204)Online publication date: 10-Jun-2022
https://dl.acm.org/doi/10.1145/3514221.3526056
Krakowski FRuhland FSchottner M(2021)Infinileap: Modern High-Performance Networking for Distributed Java Applications based on RDMA2021 IEEE 27th International Conference on Parallel and Distributed Systems (ICPADS)10.1109/ICPADS53394.2021.00087(652-659)Online publication date: Dec-2021
https://doi.org/10.1109/ICPADS53394.2021.00087
Zhang ZLiu ZJiang QChen JAn H(2021)RDMA-Based Apache Storm for High-Performance Stream Data ProcessingInternational Journal of Parallel Programming10.1007/s10766-021-00696-0Online publication date: 18-Mar-2021
https://doi.org/10.1007/s10766-021-00696-0
Zhang ZLiu ZJiang QWu ZChen JAn H(2021)RDMA-Based Apache Storm for High-Performance Stream Data ProcessingNetwork and Parallel Computing10.1007/978-3-030-79478-1_24(276-287)Online publication date: 23-Jun-2021
https://doi.org/10.1007/978-3-030-79478-1_24
Ruhland FKrakowski FSchottner M(2020)Performance analysis and evaluation of Java-based InfiniBand Solutions2020 19th International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC51135.2020.00013(20-28)Online publication date: Jul-2020
https://doi.org/10.1109/ISPDC51135.2020.00013
Krakowski FRuhland FSchottner M(2020)Neutrino: Efficient InfiniBand Access for Java Applications2020 19th International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC51135.2020.00012(12-19)Online publication date: Jul-2020
https://doi.org/10.1109/ISPDC51135.2020.00012
Nothaas SBeineke KSchoettner M(2019)Leveraging InfiniBand for Highly Concurrent Messaging in Java Applications2019 18th International Symposium on Parallel and Distributed Computing (ISPDC)10.1109/ISPDC.2019.00013(74-83)Online publication date: Jun-2019
https://doi.org/10.1109/ISPDC.2019.00013
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten