skip to main content
10.1145/3005745.3005756acmconferencesArticle/Chapter ViewAbstractPublication PagescommConference Proceedingsconference-collections
research-article

FreeFlow: High Performance Container Networking

Published: 09 November 2016 Publication History

Abstract

With the tremendous popularity gained by container technology, many applications are being containerized: splitting into numerous containers connected by networks. However, current container networking solutions have either bad performance or poor portability, which undermines the advantages of containerization. In this paper, we propose FreeFlow, a container networking solution which achieves both high performance and good portability. FreeFlow is designed according to the observation that strict isolations are unnecessary among containers trusting each other, and it can significantly boost the communication quality of containers by compromising isolation a little bit. Specifically, we enable containers on the same physical machine to communicate via shared-memory and the ones on different physical machines communicate via high performance networking options, e.g. RDMA and DPDK. Naively wrapping up all the solutions together will result in poor potability of containers and huge complexity in application development. Instead, FreeFlow leverages a network abstraction which supports all common network APIs and a centralized network orchestrator which decides how to deliver data transparently to applications in the containers.

References

[1]
Docker. http://www.docker.com/.
[2]
Kubernetes. http://kubernetes.io/.
[3]
Project calico. https://www.projectcalico.org/.
[4]
Apache Hadoop. http://hadoop.apache.org/, accessed 2016.
[5]
CoreOS. https://coreos.com/, accessed 2016.
[6]
Data plane development kit (DPDK). http://dpdk.org/, accessed 2016.
[7]
Weave Net. https://www.weave.works/, accessed 2016.
[8]
G. Ananthanarayanan, S. Kandula, A. G. Greenberg, I. Stoica, Y. Lu, B. Saha, and E. Harris. Reining in the outliers in map-reduce clusters using mantri. In USENIX OSDI, 2010.
[9]
P. Balaji, S. Narravula, K. Vaidyanathan, S. Krishnamoorthy, J. Wu, and D. K. Panda. Sockets direct protocol over infiniband in clusters: is it beneficial? In IEEE ISPASS, 2004.
[10]
J. Brandeburg. Reducing network latency in linux. In Linux Plumbers Conference, 2012.
[11]
M. Chowdhury and I. Stoica. Efficient coflow scheduling without prior knowledge. In ACM SIGCOMM, 2015.
[12]
M. Chowdhury, M. Zaharia, J. Ma, M. I. Jordan, and I. Stoica. Managing data transfers in computer clusters with orchestra. In ACM SIGCOMM, 2011.
[13]
M. Chowdhury, Y. Zhong, and I. Stoica. Efficient coflow scheduling with varys. In ACM SIGCOMM, 2014.
[14]
J. Claassen, R. Koning, and P. Grosso. Linux containers networking: Performance and scalability of kernel modules. In IEEE/IFIP NOMS, 2016.
[15]
Datadog. 8 suprising facts about real Docker adoption. https://www.datadoghq.com/docker-adoption/, 2016.
[16]
Docker. Docker community passes two billion pulls. https://blog.docker.com/2016/02/docker-hub-two-billion-pulls/, 2016.
[17]
A. Dragojevic, D. Narayanan, M. Castro, and O. Hodson. Farm: fast remote memory. In USENIX NSDI, 2014.
[18]
M. Fox, C. Kassimis, and J. Stevens. IBM's Shared Memory Communications over RDMA (SMC-R) Protocol. RFC 7609 (Informational), 2015.
[19]
FreeBDS. chroot - FreeBDS Man Pages. http://www.freebsd.org/cgi/man.cgi, FreeBDS 10.3 Rel.
[20]
B. Goglin and S. Moreaud. Knem: A generic and scalable kernel-assisted intra-node MPI communication framework. Journal of Parallel and Distributed Computing, 73(2), 2013.
[21]
D. Goldenberg, M. Kagan, R. Ravid, and M. S. Tsirkin. Zero copy sockets direct protocol over infiniband-preliminary implementation and performance analysis. In 13th Symposium on High Performance Interconnects (HOTI'05), pages 128-137. IEEE, 2005.
[22]
P. S. I. Group. Single root I/O virtualization. http://pcisig.com/specifications/iov/single_root/, accessed 2016.
[23]
S. Hefty. Rsockets. In OpenFabris International Workshop, 2012.
[24]
B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi, A. D. Joseph, R. H. Katz, S. Shenker, and I. Stoica. Mesos: A platform for fine-grained resource sharing in the data center. In USENIX NSDI, 2011.
[25]
T. Hoefler, J. Dinan, D. Buntinas, P. Balaji, B. Barrett, R. Brightwell, W. D. Gropp, V. Kale, and R. Thakur. Mpi + mpi: A new hybrid approach to parallel programming with mpi plus shared memory. Computing, 95, 2013.
[26]
J. Hwang, K. Ramakrishnan, and T. Wood. Netvm: high performance and flexible networking using virtualization on commodity platforms. IEEE Transactions on Network and Service Management, 12(1), 2015.
[27]
Iron.io. Docker in production - what we've learned launching over 300 million containers. https://www.iron.io/docker-in-production-what-weve-learned/, 2014.
[28]
M. Li, D. G. Andersen, A. J. Smola, and K. Yu. Communication efficient distributed machine learning with the parameter server. In Advances in Neural Information Processing Systems, 2014.
[29]
L. Liss. Containing RDMA and high performance computing. In ContainerCon, 2015.
[30]
J. Liu, J. Wu, and D. K. Panda. High performance RDMA-based MPI implementation over infiniband. International Journal of Parallel Programming, 32(3), 2004.
[31]
A. Madhavapeddy, T. Leonard, M. Skjegstad, T. Gazagnaire, et al. Jitsu: Just-in-time summoning of unikernels. In USENIX NSDI, 2015.
[32]
J. Martins, M. Ahmed, C. Raiciu, V. Olteanu, M. Honda, R. Bifulco, and F. Huici. Clickos and the art of network function virtualization. In USENIX NSDI, 2014.
[33]
Mellanox. RDMA aware networks programming user manual. http://www.mellanox.com/, Rev 1.7.
[34]
D. Merkel. Docker: Lightweight linux containers for consistent development and deployment. Linux J., 2014(239), 2014.
[35]
R. Rabenseifner, G. Hager, and G. Jost. Hybrid mpi/openmp parallel programming on clusters of multi-core smp nodes. In Euromicro International Conference on Parallel, Distributed and Network-based Processing, 2009.
[36]
A. Ranadive and B. Davda. Toward a paravirtual vRDMA device for VMware ESXi guests. VMware Technical Journal, Winter 2012, 1(2), 2012.
[37]
L. Rizzo. Netmap: a novel framework for fast packet i/o. In USENIX Security Symposium, 2012.
[38]
L. Rizzo and G. Lettieri. Vale, a switched ethernet for virtual machines. In ACM CoNEXT, 2012.
[39]
V. K. Vavilapalli, A. C. Murthy, C. Douglas, S. Agarwal, et al. Apache hadoop yarn: Yet another resource negotiator. In ACM Symposium on Cloud Computing, 2013.
[40]
J. Wang, K.-L. Wright, and K. Gopalan. Xenloop: A transparent high performance inter-vm network loopback. In ACM HPDC, 2008.
[41]
Y. Zhu, H. Eran, D. Firestone, C. Guo, et al. Congestion control for large-scale RDMA deployments. In ACM SIGCOMM, volume 45, 2015.

Cited By

View all
  • (2024)Hyperion: Hardware-Based High-Performance and Secure System for Container NetworksIEEE Transactions on Cloud Computing10.1109/TCC.2024.340317512:3(844-858)Online publication date: Jul-2024
  • (2024)Rethinking and Optimizing Workload Redistribution in Large-scale Internet Data Centers2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682613(1-10)Online publication date: 19-Jun-2024
  • (2024)UniNet: Accelerating the Container Network Data Plane in IaaS Clouds2024 IEEE 17th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD62652.2024.00023(115-127)Online publication date: 7-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HotNets '16: Proceedings of the 15th ACM Workshop on Hot Topics in Networks
November 2016
217 pages
ISBN:9781450346610
DOI:10.1145/3005745
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2016

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

HotNets-XV
Sponsor:

Acceptance Rates

HotNets '16 Paper Acceptance Rate 30 of 108 submissions, 28%;
Overall Acceptance Rate 110 of 460 submissions, 24%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)49
  • Downloads (Last 6 weeks)3
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Hyperion: Hardware-Based High-Performance and Secure System for Container NetworksIEEE Transactions on Cloud Computing10.1109/TCC.2024.340317512:3(844-858)Online publication date: Jul-2024
  • (2024)Rethinking and Optimizing Workload Redistribution in Large-scale Internet Data Centers2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682613(1-10)Online publication date: 19-Jun-2024
  • (2024)UniNet: Accelerating the Container Network Data Plane in IaaS Clouds2024 IEEE 17th International Conference on Cloud Computing (CLOUD)10.1109/CLOUD62652.2024.00023(115-127)Online publication date: 7-Jul-2024
  • (2024)High-performance microservice differentiated domain communication technologyComputing10.1007/s00607-023-01232-3106:11(3503-3535)Online publication date: 13-Jan-2024
  • (2023)On Optimizing Traffic Scheduling for Multi-replica Containerized MicroservicesProceedings of the 52nd International Conference on Parallel Processing10.1145/3605573.3605646(358-368)Online publication date: 7-Aug-2023
  • (2023)Docker Networking: A Security Review2023 7th International Conference on Trends in Electronics and Informatics (ICOEI)10.1109/ICOEI56765.2023.10126070(624-629)Online publication date: 11-Apr-2023
  • (2023)Energy optimized container placement for cloud data centers: a meta-heuristic approachThe Journal of Supercomputing10.1007/s11227-023-05462-280:1(98-140)Online publication date: 22-Jun-2023
  • (2022)EZPath: Expediting Container Network Traffic via Programmable Switches2022 IFIP Networking Conference (IFIP Networking)10.23919/IFIPNetworking55013.2022.9829818(1-8)Online publication date: 13-Jun-2022
  • (2021)Parallelizing packet processing in container overlay networksProceedings of the Sixteenth European Conference on Computer Systems10.1145/3447786.3456241(261-276)Online publication date: 21-Apr-2021
  • (2021)Joint Task Scheduling and Containerizing for Efficient Edge ComputingIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2021.305944732:8(2086-2100)Online publication date: 1-Aug-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media