Kun Suo
ABSTRACT
Containers, an emerging service to manage and deploy applications into isolated boxes, are quickly increasing in popularity in the cloud and edge computing. In order to provide connectivity among multiple hosts, cloud providers adopt overlay networks, which not only impose significant overhead in throughput and latency in containerized applications, but also consume more CPU resources of the system. Through profiling and code analysis, this paper reveals that the overwhelming interrupts, as well as its load imbalance in the kernel processing contribute to the inefficiency of the container overlay networks. Specifically, every packet in container networks might raise multiple software interrupts compared to that in VM networks. Our results indicate that the container network throughput drops 2/3 and the tail latency increases more than 37 times if the interrupt overhead is not well optimized.
- [n.d.]. 4 Container Usage Takeaways from the 2019 Sysdig Report. https://developers.redhat.com/blog/2019/11/08/4-container-usage-takeaways-from-the-2019-sysdig-report/.Google Scholar
- [n.d.]. Amazon Elastic Container Service. https://aws.amazon.com/ecs/.Google Scholar
- [n.d.]. Apache Mesos. https://mesos.apache.org/.Google Scholar
- [n.d.]. Azure Container Service. https://azure.microsoft.com/en-us/services/container-service/.Google Scholar
- [n.d.]. Cloudsuite. https://cloudsuite.ch.Google Scholar
- [n.d.]. Docker Swarm. https://docs.docker.com/engine/swarm/.Google Scholar
- [n.d.]. Everything at Google Runs in Containers. https://www.infoq.com/news/2014/06/everything-google-containers/.Google Scholar
- [n.d.]. Flame Graph. https://github.com/brendangregg/FlameGraph.Google Scholar
- [n.d.]. Google Cloud Container. https://cloud.google.com/containers/.Google Scholar
- [n.d.]. iPerf. https://iperf.fr/.Google Scholar
- [n.d.]. Irqbalance. https://github.com/Irqbalance/irqbalance.Google Scholar
- [n.d.]. Kubernetes. https://kubernetes.io/.Google Scholar
- [n.d.]. Memcached. https://memcached.org/.Google Scholar
- [n.d.]. Netperf. http://www.netperf.org/.Google Scholar
- [n.d.]. Sockperf. https://github.com/Mellanox/sockperf.Google Scholar
- B. Burns and D. Oppenheimer. 2016. Design Patterns For Container-based Distributed Systems. In Proceedings of USENIX Workshop on Hot Topics in Cloud Computing (HotCloud). Denver, USA.Google Scholar
- W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. 2015. An Updated Performance Comparison of Virtual Machines and Linux Containers. In Proceedings of IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). Philadelphia, USA.Google Scholar
- S. Han, S. Marshall, B. Chun, and S. Ratnasamy. 2012. MegaPipe: A New Programming Interface for Scalable Network I/O. In Proceedings of the 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI). Hollywood, CA, USA.Google Scholar
- E. Jeong, S. Woo, M. Jamshed, H. Jeong, S. Ihm, D. Han, and K. Park. 2014. mTCP: a Highly Scalable User-level TCP Stack for Multicore Systems.. In Proceedings of USENIX Symposium on Networked Systems Design and Implementation (NSDI). Seattle, WA, USA.Google Scholar
- J. Khalid, E. Rozner, W. Felter, C. Xu, K. Rajamani, A. Ferreira, and A. Akella. 2018. Iron: Isolating Network-based CPU in Container Environments. In Proceedings of USENIX Symposium on Networked Systems Design and Implementation (NSDI). Renton, WA, USA.Google Scholar
- D. Merkel. 2014. Docker: Lightweight Linux Containers for Consistent Development and Deployment. In Linux Journal.Google Scholar
- A. Panda, M. Sagiv, and S. Shenker. 2017. Verification in the Age of Microservices. In Proceedings of USENIX Workshop on Hot Topics in Operating Systems (HotOS). Vancouver, Canada.Google Scholar
- S. Peter, J. Li, I. Zhang, D. Ports, D. Woos, A. Krishnamurthy, T. Anderson, and T. Roscoe. 2014. Arrakis: the Operating System Is The Control Plane. In Proceedings of USENIX Symposium on Operating System Design and Implementation (OSDI). Broomfield, CO, USA.Google Scholar
- P. Sharma, L. Chaufournier, P. Shenoy, and Y. Tay. 2016. Containers and Virtual Machines at Scale: A Comparative Study. In Proceedings of ACM/IFIP Middleware conference (Middleware). Trento, Italy.Google Scholar
- K. Suo, J. Rao, L. Cheng, and F. C. M. Lau. 2016. Time Capsule: Tracing Packet Latency across Different Layers in Virtualized Systems. In Proceedings of the 7th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys). Hong Kong, China.Google ScholarDigital Library
- K. Suo, J. Rao, H. Jiang, and W. Srisa-an. 2018. Characterizing and Optimizing Hotspot Parallel Garbage Collection On Multicore Systems. In Proceedings of the Thirteenth European Conference on Computer Systems (EuroSys). Porto, Portugal.Google Scholar
- K. Suo, Y. Zhao, W. Chen, and J. Rao. 2018. An Analysis and Empirical Study of Container Networks. In Proceedings of IEEE International Conference on Computer Communications (INFOCOM). Honolulu, HI, USA.Google Scholar
- K. Suo, Y. Zhao, W. Chen, and J. Rao. 2018. vNetTracer: Efficient and Programmable Packet Tracing in Virtualized Networks. In Proceedings of International Conference on Distributed Computing Systems (ICDCS). Vienna, Austria.Google Scholar
- K. Suo, Y. Zhao, J. Rao, L. Cheng, X. Zhou, and F. Lau. 2017. Preserving I/O Prioritization in Virtualized OSes. In Proceedings of the 2017 Symposium on Cloud Computing (SoCC). Santa Clara, California, USA.Google Scholar
- T. Yu, S. Noghabi, S. Raindel, H. Liu, J. Padhye, and V. Sekar. 2016. FreeFlow: High Performance Container Networking. In Proceedings of ACM Workshop on Hot Topics in Networks (HotNet). Atlanta, GA, USA.Google Scholar
- D. Zhuo, K. Zhang, Y. Zhu, H. Liu, M. Rockett, A. Krishnamurthy, and T. Anderson. 2019. Slim: OS Kernel Support for a Low-Overhead Container Overlay Network. In Proceedings of USENIX Symposium on Networked Systems Design and Implementation (NSDI). Boston, MA, USA.Google Scholar
Index Terms
- Characterizing networking performance and interrupt overhead of container overlay networks
Recommendations
Performance of Container Networking Technologies
HotConNet '17: Proceedings of the Workshop on Hot Topics in Container Networking and Networked SystemsContainer networking is now an important part of cloud virtualization architectures. It provides network access for containers by connecting both virtual and physical network interfaces. The performance of container networking has multiple dependencies, ...
AWS EC2 vs. Joyent's Triton: A Comparison of Docker Container-hosting Platforms
ScienceCloud '17: Proceedings of the 8th Workshop on Scientific Cloud ComputingContainers, and in particular Docker, have emerged as promising addition and in some cases, alternative to virtual machines (VMs) for application deployment in cloud environments. Docker containers enable massive scalability and rapid deployment of ...
Container stowage pre-planning: using search to generate solutions, a case study
Container-ships are vessels possessing an internal structure that facilitates the handling of containerised cargo. At each port along the vessel's journey, containers destined for those ports are unloaded and additional containers destined for ...
Comments