skip to main content
10.1145/2837030.2837036acmotherconferencesArticle/Chapter ViewAbstractPublication PagesaintecConference Proceedingsconference-collections
research-article

Toward Fast and Scalable Key-Value Stores Based on User Space TCP/IP Stack

Published: 18 November 2015 Publication History

Abstract

Key-Value Stores (KVS) are wildly prevalent in Web systems and services, and it is well-known that KVS systems exchange huge amount of small messages. Such workload unveils a new performance issue for TCP/IP processing. User-space networking such as Intel DPDK is a promising technology that addresses this issue by leveraging multi-core architecture and application specific processing without the intervention of the kernel. This paper shows the effects of user-space TCP/IP stack that works on top of Intel DPDK for high performance KVS systems. We have compared the performance of short message processing on an HTTP server and two KVS servers, Redis and memcached, between the Linux kernel TCP/IP stack and a user-space TCP stack over 10 Gigabit Ethernet link. We have used mTCP with Intel DPDK as a user-space TCP/IP stack. Our experimental results show that an mTCP-based KVS server with a single thread improves the performance by up to 35 %; the performance of an mTCP-based server increases as the number of threads increases and an mTCP server finally outperforms up to 5.5 times faster than a standard TCP server. Our observation leads to optimistic conclusions on the feasibility of a fast and scalable KVS server utilizing a user-space TCP/IP stack.

References

[1]
B. Atikoglu, Y. Xu, E. Frachtenberg, S. Jiang, and M. Paleczny. Workload Analysis of a Large-scale Key-value Store. In Proceedings of the 12th ACM SIGMETRICS/PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS '12, pages 53--64, 2012.
[2]
Cloudius Systems. SeaStar. http://www.seastar-project.org.
[3]
S. Han, K. Jang, K. Park, and S. Moon. PacketShader: A GPU-accelerated Software Router. SIGCOMM Comput. Commun. Rev., 40(4):195--206, Aug. 2010.
[4]
S. Han, S. Marshall, B.-G. Chun, and S. Ratnasamy. MegaPipe: A New Programming Interface for Scalable Network I/O. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation, OSDI'12, pages 135--148. USENIX Association, 2012.
[5]
M. Honda, F. Huici, C. Raiciu, J. Araujo, and L. Rizzo. Rekindling Network Protocol Innovation with User-level Stacks. SIGCOMM Comput. Commun. Rev., 44(2):52--58, Apr. 2014.
[6]
Intel. DPDK: Data Plane Development Kit. http://dpdk.org.
[7]
E. Y. Jeong, S. Woo, M. Jamshed, H. Jeong, S. Ihm, D. Han, and K. Park. mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation, NSDI'14, pages 489--502, 2014.
[8]
D. Kegel. The C10K problem. http://www.kegel.com/c10k.html.
[9]
H. Lim, D. Han, D. G. Andersen, and M. Kaminsky. MICA: A Holistic Approach to Fast In-memory Key-value Storage. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation, NSDI'14, pages 429--444. USENIX Association, 2014.
[10]
LWN.net. The SO_REUSEPORT socket option. https://lwn.net/Articles/542629/.
[11]
K. Magoutis, M. I. Seltzer, and E. Gabber. The Case Against User-level Networking. In Procddings of the Third Workshop on Novel Uses of System Area Networks (SAN-3) (Held in conjunction with HPCA-10), Feb. 2004.
[12]
I. Marinos, R. N. M. Watson, and M. Handley. Network Stack Specialization for Performance. In Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks, HotNets-XII, pages 9:1--9:7. ACM, 2013.
[13]
N. Provos and N. Mathewson. libevent -- an event notification library. http://libevent.org.
[14]
L. Rizzo. netmap: A Novel Framework for Fast Packet I/O. In 21st USENIX Security Symposium (USENIX Security 12), pages 101--112. USENIX Association, Aug. 2012.
[15]
S. Sanfilippo. mc-benchmark -- memcache port of redis benchmark. https://github.com/antirez/mc-benchmark.
[16]
S. Sanfilippo and P. Noordhuis. Redis -- an open source, BSD licensed, advanced key-value cache and store. http://redis.io.
[17]
A. Vorobey and B. Fitzpatrick. memcached -- a distributed memory object caching system. http://memcached.org.
[18]
S. Woo, E. Jeong, S. Park, J. Lee, S. Ihm, and K. Park. Comparison of caching strategies in modern cellular backhaul networks. In Procddings of the annual international conference on Mobile systems, applications, and services (MobiSys), June 2013.

Cited By

View all
  • (2017)muMQ: A lightweight and scalable MQTT broker2017 IEEE International Symposium on Local and Metropolitan Area Networks (LANMAN)10.1109/LANMAN.2017.7972165(1-6)Online publication date: Jun-2017
  • (2016)Towards Scalable and Reliable In-Memory Storage System: A Case Study with Redis2016 IEEE Trustcom/BigDataSE/ISPA10.1109/TrustCom.2016.0255(1660-1667)Online publication date: Aug-2016

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
AINTEC '15: Proceedings of the 11th Asian Internet Engineering Conference
November 2015
77 pages
ISBN:9781450339148
DOI:10.1145/2837030
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 November 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Key-Value Stores
  2. Multi-core Systems
  3. User-space TCP/IP stack

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AINTEC '15
AINTEC '15: Asian Internet Engineering Conference
November 18 - 20, 2015
Bangkok, Thailand

Acceptance Rates

Overall Acceptance Rate 15 of 38 submissions, 39%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)46
  • Downloads (Last 6 weeks)1
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2017)muMQ: A lightweight and scalable MQTT broker2017 IEEE International Symposium on Local and Metropolitan Area Networks (LANMAN)10.1109/LANMAN.2017.7972165(1-6)Online publication date: Jun-2017
  • (2016)Towards Scalable and Reliable In-Memory Storage System: A Case Study with Redis2016 IEEE Trustcom/BigDataSE/ISPA10.1109/TrustCom.2016.0255(1660-1667)Online publication date: Aug-2016

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media