Skip to main content

Advertisement

SpringerLink
Log in

A low-overhead networking mechanism for virtualized high-performance computing systems

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The use of virtualized parallel and distributed computing systems is rapidly becoming the mainstream due to the significant benefit of high energy-efficiency and low management cost. Processing network operations in a virtual machine, however, incurs a lot of overhead from the arbitration of network devices between virtual machines, inherently by the nature of the virtualized architecture. Since data transfer between server nodes frequently occurs in parallel and distributed computing systems, the high overhead of networking may induce significant performance loss in the overall system. This paper introduces the design and implementation of a novel networking mechanism with low overhead for virtualized server nodes. By sacrificing isolation between virtual machines, which is insignificant in distributed or parallel computing systems, our approach significantly reduces the processing overhead in networking operations by up to 29% of processor load, along with up to 36% of processor cache miss. Furthermore, it improves network bandwidth by up to 8%, especially when transmitting large packets. As a result, our prototype enhances the performance of real-world workloads by up to 12% in our evaluation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Amza C, Cecchet E, Chanda A, Cox A, Elnikety S, Gil R, Marguerite J, Rajamani K, Zwaenepoel W (2002) Specification and implementation of dynamic web site benchmarks. In: Proceedings of IEEE 5th annual workshop on workload characterization

  2. Apparao P, Makineni S, Newell D (2006) Characterization of network processing overheads in Xen. In: Proceedings of IEEE int’l workshop on virtualization technology in distributed computing, Nov 2006

  3. Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauery R, Pratt I, Warfield A (2003) Xen and the Art of Virtualization. In: Proceedings of ACM symp. operating systems principles, Oct 2003, pp 164–177

  4. Belady C (2007) In the data center, power and cooling costs more than the IT equipment it supports. In: Electronics Cooling, Feb 2007

  5. Bos H, Bruijn WD, Cristea M, Nguyen T, Portokalidis G (2004) FFPF: fairly fast packet filters. In: Proceedings of 6th symp. operating systems design and implementation, Dec 2004, pp 24

  6. Deri L (2004) Improving passive packet capture: beyond device polling. In: Proceedings of int’l system administration and network engineering conf., Sep 2004

  7. Druschel P, Peterson LL (1993) Fbufs: a high-bandwidth cross-domain transfer facility. In: Proceedings of ACM symp. operating systems principles, Dec 1993, pp 189–202

  8. Enabling virtualization in the datacenter. White paper, Neterion, Jan 2007

  9. Evangelinos C, Hill CN (2008) Cloud computing for parallel scientific HPC applications: feasibility of running coupled atmosphere-ocean climate models on Amazon’s EC2. In: Proceedings of cloud computing and its applications

  10. GadelRab S (2007) 10-Gigabit ethernet connectivity for computer servers. IEEE Micro 27(3):94–105

    Article  Google Scholar 

  11. Govindan R, Anderson DP (1991) Scheduling and IPC mechanisms for continuous media. Proc ACM SIGOPS Oper Syst Rev 25(5):68–80

    Article  Google Scholar 

  12. Govindan S, Nath AR, Das A, Urgaonkar B, Sivasubramaniam A (2007) Xen and Co.: Communication-aware CPU scheduling for consolidated Xen-based hosting platforms. In: Proceedings of int’l conf. virtual execution environments, Jun 2007, pp 126–136

  13. Henning JL (2006) SPEC CPU2006 benchmark descriptions. ACM SIGARCH Comput Archit News 34(4):1–17

    Article  MathSciNet  Google Scholar 

  14. Kamp P-H, Watson RNM (2000) Jails: confining the omnipotent root. In: Proceedings of int’l system administration and networking conference, May 2000

  15. Keahey K, Figueiredo R, Fortes J, Freeman T, Tsugawa M (2008) Science clouds: early experiences in cloud computing for scientific applications. In: Proceedings of cloud computing and its applications

  16. Kim K, Kim C, Jung S-I, Shin H-S, Kim J-S (2008) Inter-domain socket communications supporting high performance and full binary compatibility on Xen. In: Proceedings of ACM SIGPLAN/SIGOPS int’l conf. virtual execution environments, Mar 2008, pp 11–20

  17. Koomey J (2006) Estimating total power consumption by servers in the U.S. and the world. Technical report, Lawrence Berkeley National Laboratory

  18. Kopytov A (2009) SysBench: a system performance benchmark. http://sysbench.sourceforge.net/

  19. Liu J, Huang W, Abali B, Panda DK (2006) High performance VMM-bypass I/O in virtual machines. In: Proceedings of 2006 USENIX annual technical conf., May 2006, pp 1–3

  20. Menon A, Cox AL, Zwaenepoel W (2006) Optimizing network virtualization in Xen. In: Proceedings of 2006 USENIX annual technical conf., Jun 2006, pp 15–28

  21. Menon A, Santos JR, Turner Y, Janakiraman G, Zwaenepoel W (2005) Diagnosing performance overheads in the xen virtual machine environment. In: Proceedings of ACM/USENIX int’l conf. virtual execution environments, Jun 2005, pp 13–23

  22. Padala P, Zhu X, Wang Z, Singhal S, Shin KG (2007) Performance evaluation of virtualization technologies for server consolidation. Technical Report HPL-2007-59, HP

  23. Price D, Tucker A (2004) Solaris zones: operating systems support for consolidating commercial workloads. In: Proceedings of 18th large installation system administration conf., Nov. 2004, pp 241–254

  24. Raj H, Ganev I, Schwan K, Xenidis J (2006) Self-virtualized I/O: high performance, scalable I/O virtualization in multi-core systems. Technical Report GIT-CERCS-06-02, CERCS, Georgia Tech

  25. Santos JR Janakiraman G, Turner Y (2007) Xen network I/O: Performance Analysis and Opportunities for Improvement. Xen Summit Spring 2007. http://xen.xensource.com/files/xensummit_4/NetworkIO_Santos.pdf, Apr

  26. Singh R (2007) Server virtualization and consolidation — a case study. White paper. http://www.ibm.com/support/techdocs, IBM

  27. Soltesz S, Pötzl H, Fiuczynski ME, Bavier A, Peterson L (2007) Container-based operating system virtualization: a scalable, high-performance alternative to hypervisors. In: Proceedings of ACM SIGOPS/Eurosys European conf. on computer systems, Mar. 2007, pp 275–287

  28. Sugerman J, Venkitachalam G, Lim B-H (2001) Virtualizing I/O devices on VMware workstation’s hosted virtual machine monitor. In: Proceedings of 2001 USENIX annual technical conf., Jun 2001, pp 1–14

  29. Waldspurger CA (2002) Memory resource management in VMware ESX server. In: Proceedings of symp. operating systems design and implementation, Dec 2002, pp 181–194

  30. Virtual Machine Device Queues. White paper, Intel, 2007

  31. Willmann P, Shafer J, Carr D, Menon A, Rixner S, Cox AL, Zwaenepoel W (2007) Concurrent direct network access for virtual machine monitors. In: Proceedings of IEEE int’l symp. high performance computer architecture, Feb 2007, pp. 306–317

  32. Zhang X, McIntosh S, Rohatgi P, Griffin JL (2007) XenSocket: a high-throughput interdomain transport for virtual machines. In: Proceedings of ACM/IFIP/USENIX int’l middleware conf., Aug 2007, pp 184–203

Download references

Author information

Authors and Affiliations

  1. CS Dept., KAIST, Yuseong-gu, Daejeon, Republic of Korea

    Jae-Wan Jang & Heeseung Jo

  2. School of ECE, UNIST, Ulju-gun, Ulsan, Republic of Korea

    Euiseong Seo

  3. School of ICE, Sungkyunkwan University, Jangan-gu, Suwon, Republic of Korea

    Jin-Soo Kim

Authors
  1. Jae-Wan Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Euiseong Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heeseung Jo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin-Soo Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin-Soo Kim.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jang, JW., Seo, E., Jo, H. et al. A low-overhead networking mechanism for virtualized high-performance computing systems. J Supercomput 59, 443–468 (2012). https://doi.org/10.1007/s11227-010-0444-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-010-0444-9

Keywords

Search

Navigation