Myoungsung You
ABSTRACT
With the increasing adoption of containerization in cloud services, container networking has become a critical concern, as it enables the agile deployment of microservices but also introduces new vulnerabilities susceptible to network attacks, posing a threat to container environments. While several security solutions have been introduced to address this concern, they unfortunately exhibit significant shortcomings, including security vulnerabilities and limited performance. We thus propose Helios, a novel hardware-based network security extension that addresses the security and performance limitations in existing solutions. Leveraging a smartNIC, Helios enhances both the security and performance facets of container networking through two key mechanisms: (i) the establishment of physically isolated container communication channels and (ii) the network security engines fully offloaded to the smartNIC. Our evaluation shows that Helios mitigates various network threats initiated from both container- and host-side while performing up to 3x faster than the existing solutions in container communication.
- 2008. PCI-SIG Single Root I/O Virtualization (SR-IOV) Support in Intel® Virtualization Technology for Connectivity. https://www.intel.com/content/www/us/en/pci-express/pci-sig-single-root-io-virtualization-support-in-virtualization-technology-for-connectivity-paper.html.Google Scholar
- 2013. Namespaces in Operation, Part 1: Namespaces Overview. https://lwn.net/Articles/531114/.Google Scholar
- 2014. wrk -- a HTTP Benchmarking Tool. https://github.com/wg/wrk.Google Scholar
- 2015. Kubernetes Performance Measurements and Roadmap. https://kubernetes.io/blog/2015/09/kubernetes-performance-measurements-and/.Google Scholar
- 2018. Flask Docker Container Image. https://hub.docker.com/r/jcdemo/flaskapp.Google Scholar
- 2019. CVE-2019-8341. https://nvd.nist.gov/vuln/detail/CVE-2019-8341/.Google Scholar
- 2020. CVE-2020-11100. https://nvd.nist.gov/vuln/detail/CVE-2020-11100/.Google Scholar
- 2021. State of Kubernetes Security Report. https://thechief.io/c/editorial/state-of-kubernetes-security-report.Google Scholar
- 2022. 7 Most Infamous Cloud Security Breaches. https://blog.storagecraft.com/7-infamous-cloud-security-breaches/.Google Scholar
- 2022. Amazon Web Services. https://aws.amazon.com/.Google Scholar
- 2023. AppArmor, Linux Kernel Security Module. https://apparmor.net/.Google Scholar
- 2023. bridge(8) --- Linux manual page. https://man7.org/linux/manpages/man8/bridge.8.html.Google Scholar
- 2023. Calio-felix. https://docs.projectcalico.org/reference/felix/.Google Scholar
- 2023. Cilim Envoy Extension. https://docs.cilium.io/en/v1.13/security/network/proxy/envoy/.Google Scholar
- 2023. Cilium. https://www.cilium.io/.Google Scholar
- 2023. Cilium-agent. https://docs.cilium.io/en/stable/cmdref/cilium-agent/.Google Scholar
- 2023. CNI: The container network interface. https://www.cni.dev/.Google Scholar
- 2023. Docker. https://www.docker.com.Google Scholar
- 2023. Docker host networking. https://docs.docker.com/network/host/.Google Scholar
- 2023. DockerHub: envoyproxy/envoy. https://hub.docker.com/r/envoyproxy/envoy.Google Scholar
- 2023. DockerHub: hashicorp/boundary. https://hub.docker.com/r/hashicorp/boundary.Google Scholar
- 2023. DockerHub: sysdig. https://hub.docker.com/r/sysdig/sysdig.Google Scholar
- 2023. eBPF Introduction, Tutorials. https://docs.cilium.io/en/stable/bpf/.Google Scholar
- 2023. Flannel-d. https://github.com/flannel-io/flannel.Google Scholar
- 2023. Google Cloud Platform (GCP). https://cloud.google.com/.Google Scholar
- 2023. HAProxy ingress controler. https://haproxy-ingress.github.io/.Google Scholar
- 2023. Hewlett Packard Enterprise. Netperf. https://hewlettpackard.github.io/netperf/.Google Scholar
- 2023. Host network driver. https://docs.docker.com/network/drivers/host/.Google Scholar
- 2023. iPerf. Network Bandwidth Measurement Tool. https://iperf.fr/iperf-download.php.Google Scholar
- 2023. Kubernetes. https://kubernetes.io.Google Scholar
- 2023. Kubernetes API Watcher Design. https://docs.openstack.org/kuryr/0.2.0/devref/k8s_api_watcher_design.html.Google Scholar
- 2023. Kubernetes: Considerations for large clusters. https://kubernetes.io/docs/setup/best-practices/cluster-large/.Google Scholar
- 2023. Kubernetes Privilege Escalation. https://i.blackhat.com/USA-22/Thursday/US-22-Avrahami-Kubernetes-Privilege-Escalation-Container-Escape-Cluster-Admin.pdf.Google Scholar
- 2023. Linux SYSSTAT. http://sebastien.godard.pagesperso-orange.fr/.Google Scholar
- 2023. Microsoft Azure. https://azure.microsoft.com/.Google Scholar
- 2023. Netronome Agilo CX smartNIC 2x40GbE. https://www.netronome.com/media/documents/PB_NFP-4000-7-20.pdf.Google Scholar
- 2023. Nginx Docker Container. https://hub.docker.com/_/nginx.Google Scholar
- 2023. OpenVPN Access Server. https://hub.docker.com/r/mace/openvpn-as.Google Scholar
- 2023. Project Calico. https://www.projectcalico.org/.Google Scholar
- 2023. Redis Docker Container. https://hub.docker.com/_/redis.Google Scholar
- 2023. Service | Kubernetes. https://kubernetes.io/docs/concepts/services-networking/service/.Google Scholar
- 2023. TCPdump manpage. https://www.tcpdump.org/manpages/.Google Scholar
- 2023. The Istio service mesh. https://istio.io/.Google Scholar
- 2023. The Linked servie mesh. https://linkerd.io/.Google Scholar
- 2023. veth -- Virtual Ethernet Device. https://man7.org/linux/manpages/man4/veth.4.html/.Google Scholar
- Ali AlSabeh, Elie Kfoury, Jorge Crichigno, and Elias Bou-Harb. 2022. P4DDPI: Securing P4-Programmable Data Plane Networks via DNS Deep Packet Inspection. In Proceedings of the 2022 Network and Distributed System Security (NDSS) Symposium. 1ś7.Google ScholarCross Ref
- Kelly Brady, Seung Moon, Tuan Nguyen, and Joel Coffman. 2020. Docker Container Security in Cloud Computing. In In Proceedings of Annual Computing and Communication Workshop and Conference. 975--980.Google Scholar
- Gerald Budigiri, Christoph Baumann, Jan Tobias Mühlberg, Eddy Truyen, and Wouter Joosen. 2021. Network Policies in Kubernetes: Performance Evaluation and Security Analysis. In In proceedings of Joint European Conference on Networks and Communications & 6G Summit. 407--412.Google Scholar
- Pubali Datta, Prabuddha Kumar, Tristan Morris, Michael Grace, Amir Rahmati, and Adam Bates. 2020. Valve: Securing Function Workflows on Serverless Computing Platforms. In Proceedings of the Web Conference 2020. 939--950.Google ScholarDigital Library
- Ana Duarte and Nuno Antunes. 2018. An Empirical Study of Docker Vulnerabilities and of Static Code Analysis Applicability. In In Proceedings of Latin-American Symposium on Dependable Computing. 27--36.Google Scholar
- William Findlay, David Barrera, and Anil Somayaji. 2021. BPFContain: Fixing the Soft Underbelly of Container Security. arXiv preprint arXiv:2102.06972 (2021).Google Scholar
- Seyedhamed Ghavamnia, Tapti Palit, Azzedine Benameur, and Michalis Polychronakis. 2020. Confine: Automated System Call Policy Generation for Container Attack Surface Reduction. In Proceedings of International Symposium on Research in Attacks, Intrusions and Defenses. 443--458.Google Scholar
- Joel Hypolite, John Sonchack, Shlomo Hershkop, Nathan Dautenhahn, André DeHon, and Jonathan M Smith. 2020. DeepMatch: practical deep packet inspection in the data plane using network processors. In Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies. 336--350.Google ScholarDigital Library
- Theo Jepsen, Daniel Alvarez, Nate Foster, Changhoon Kim, Jeongkeun Lee, Masoud Moshref, and Robert Soulé. 2019. Fast string searching on pisa. In Proceedings of ACM Symposium on SDN Research. 21--28.Google ScholarDigital Library
- Jakub Kicinski and Nicolaas Viljoen. 2016. eBPF Hardware Offload to SmartNICs: cls bpf and XDP. Proceedings of netdev 1 (2016).Google Scholar
- Abhinav Kommula, Yen-Hung Frank Hu, Mary Ann Hoppa, and Samuel Olatunbosun. 2020. Machine Learning Techniques to Enhance Container Network Security. In In proceedings of International Conference on Computational Science and Computational Intelligence. 622--627.Google ScholarCross Ref
- Lingguang Lei, Jianhua Sun, Kun Sun, Chris Shenefiel, Rui Ma, Yuewu Wang, and Qi Li. 2017. SPEAKER: Split-phase execution of application containers. In Proceedings of International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment.Google ScholarCross Ref
- Wubin Li, Yves Lemieux, Jing Gao, Zhuofeng Zhao, and Yanbo Han. 2019. Service mesh: Challenges, state of the art, and future research opportunities. In Proceedings of IEEE International Conference on Service-Oriented System Engineering. 122--1225.Google ScholarCross Ref
- Xing Li, Xue Leng, and Yan Chen. 2021. Securing Serverless Computing: Challenges, Solutions, and Opportunities. arXiv preprint arXiv:2105.12581 (2021).Google Scholar
- Xin Lin, Lingguang Lei, Yuewu Wang, Jiwu Jing, Kun Sun, and Quan Zhou. 2018. A measurement study on linux container security: Attacks and countermeasures. In Proceedings of Annual Computer Security Applications Conference. 418--429.Google ScholarDigital Library
- Coleman Link, Jesse Sarran, Garegin Grigoryan, Minseok Kwon, M Mustafa Rafique, and Warren R Carithers. 2019. Container Orchestration by Kubernetes for RDMA Networking. In Proceedings of IEEE International Conference on Network Protocols. 1--2.Google ScholarCross Ref
- Chang Liu, Longtao He, Gang Xiong, Zigang Cao, and Zhen Li. 2019. Fs-net: A flow sequence network for encrypted traffic classification. In IEEE INFOCOM 2019-IEEE Conference On Computer Communications. IEEE, 1171--1179.Google ScholarDigital Library
- Antony Martin, Simone Raponi, Théo Combe, and Roberto Di Pietro. 2018. Docker Ecosystem -- Vulnerability Analysis. Computer Communications 122 (2018), 30--43.Google ScholarCross Ref
- Jaehyun Nam, Seungsoo Lee, Phillip Porras, Vinod Yegneswaran, and Seungwon Shin. 2022. Secure Inter-Container Communications Using XDP/eBPF. IEEE/ACM Transactions on Networking 31, 2 (2022), 934--947.Google ScholarDigital Library
- Jaehyun Nam, Seungsoo Lee, Hyunmin Seo, Phil Porras, Vinod Yegneswaran, and Seungwon Shin. 2020. BASTION: A Security Enforcement Network Stack for Container Networks. In Proceedings of USENIX Annual Technical Conference. 81--95.Google Scholar
- Salvatore Pontarelli, Roberto Bifulco, Marco Bonola, Carmelo Cascone, Marco Spaziani, Valerio Bruschi, Davide Sanvito, Giuseppe Siracusano, Antonio Capone, Michio Honda, et al. 2019. Flowblaze: Stateful packet processing in hardware. In Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2019. USENIX ASSOC, 531--547.Google Scholar
- Jamal Hadi Salim. 2015. Linux traffic control classifier-action subsystem architecture. Proceedings of Netdev 0.1 (2015).Google Scholar
- Meng Shen, Jinpeng Zhang, Liehuang Zhu, Ke Xu, and Xiaojiang Du. 2021. Accurate decentralized application identification via encrypted traffic analysis using graph neural networks. IEEE Transactions on Information Forensics and Security 16 (2021), 2367--2380.Google ScholarCross Ref
- Sari Sultan, Imtiaz Ahmad, and Tassos Dimitriou. 2019. Container Security: Issues, Challenges, and the Road Ahead. IEEE Access 7 (2019), 52976--52996.Google ScholarCross Ref
- Yuqiong Sun, David Safford, Mimi Zohar, Dimitrios Pendarakis, Zhongshu Gu, and Trent Jaeger. 2018. Security Namespace: Making Linux Security Frameworks Available to Containers. In Proceedings of USENIX Security Symposium. 1423--1439.Google Scholar
- Kun Suo, Yong Zhao, Wei Chen, and Jia Rao. 2018. An analysis and empirical study of container networks. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications. IEEE, 189--197.Google ScholarDigital Library
- Linyih Teng, Chi-Hsiang Hung, and Charles H-P Wen. 2022. P4SF: A High-Performance Stateful Firewall on Commodity P4-Programmable Switch. In NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium. IEEE, 1--5.Google ScholarDigital Library
- Wei Wang, Ming Zhu, Jinlin Wang, Xuewen Zeng, and Zhongzhen Yang. 2017. End-to-end encrypted traffic classification with one-dimensional convolution neural networks. In 2017 IEEE international conference on intelligence and security informatics (ISI). IEEE, 43--48.Google ScholarDigital Library
- Jinli Yan, Lu Tang, Junnan Li, Xiangrui Yang, Wei Quan, Hongyi Chen, and Zhigang Sun. 2019. UniSec: a unified security framework with SmartNIC acceleration in public cloud. In Proceedings of the ACM Turing Celebration Conference-China. 1--6.Google ScholarDigital Library
- Zirak Zaheer, Hyunseok Chang, Sarit Mukherjee, and Jacobus Van der Merwe. 2019. Eztrust: Network-independent Zero-trust Perimeterization for Microservices. In Proceedings of the Symposium on SDN Research. 49--61.Google ScholarDigital Library
Index Terms
- HELIOS: Hardware-assisted High-performance Security Extension for Cloud Networking
Recommendations
CNTC: A Container Aware Network Traffic Control Framework
Green, Pervasive, and Cloud ComputingAbstractAs a lightweight virtualization technology, containers are attracting much attention and widely deployed in the cloud data centers. To provide consistent and reliable performance, cloud providers should ensure resource isolation since each host ...
SuperNIC: An FPGA-Based, Cloud-Oriented SmartNIC
FPGA '24: Proceedings of the 2024 ACM/SIGDA International Symposium on Field Programmable Gate ArraysWith CPU scaling slowing down in today's data centers, more functionalities are being offloaded from the CPU to auxiliary devices. One such device is the SmartNIC, which is being increasingly adopted in data centers. In today's cloud environment, VMs on ...
Janus: An Experimental Reconfigurable SmartNIC with P4 Programmability and SDN Isolation
FPGA '23: Proceedings of the 2023 ACM/SIGDA International Symposium on Field Programmable Gate ArraysDisparate deployment models of cloud computing pose varying requirements on cloud infrastructure components such as networking, storage, provisioning, and security. Infrastructure providers need to study these and often create custom infrastructure ...
Comments