Skip to main content
SpringerLink
Log in

Throughput and Latency of Virtual Switching with Open vSwitch: A Quantitative Analysis

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Virtual switches, like Open vSwitch, have emerged as an important part of today’s data centers. They connect interfaces of virtual machines and provide an uplink to the physical network via network interface cards. We discuss usage scenarios for virtual switches involving physical and virtual network interfaces. We present extensive black-box tests to quantify the throughput and latency of software switches with emphasis on the market leader, Open vSwitch. Finally, we explain the observed effects using white-box measurements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. The “New API” network interface was introduced with this kernel version.

  2. Lower than the previously stated figure of 1.88 Mpps due to active profiling.

References

  1. Angrisani, L., Ventre, G., Peluso, L., Tedesco, A.: Measurement of processing and queuing delays introduced by an open-source router in a single-hop network. IEEE Trans. Instrum. Meas. 55(4), 1065–1076 (2006)

    Article  Google Scholar 

  2. Beifuß, A., Raumer, D., Emmerich, P., Runge, T.M., Wohlfart, F., Wolfinger, B.E., Carle, G.: A study of networking software induced latency. In: 2nd International Conference on Networked Systems 2015. Cottbus, Germany (2015)

  3. Bianco, A., Birke, R., Giraudo, L., Palacin, M.: Openflow switching: data plane performance. In: International Conference on Communications (ICC). IEEE (2010)

  4. Bolla, R., Bruschi, R.: Linux software router: data plane optimization and performance evaluation. J. Netw. 2(3), 6–17 (2007)

    Google Scholar 

  5. Bradner, S., McQuaid, J.: Benchmarking methodology for network interconnect devices. RFC 2544 (Informational) (1999)

  6. Cardigliano, A., Deri, L., Gasparakis, J., Fusco, F.: vPFRING: towards wirespeed network monitoring using virtual machines. In: ACM Internet Measurement Conference (2011)

  7. Deri, L.: nCap: Wire-speed packet capture and transmission. In: IEEE Workshop on End-to-End Monitoring Techniques and Services, pp. 47–55 (2005)

  8. Dobrescu, M., Argyraki, K., Ratnasamy, S.: Toward predictable performance in software packet-processing platforms. In: USENIX Conference on Networked Systems Design and Implementation (NSDI) (2012)

  9. Dobrescu, M., Egi, N., Argyraki, K., Chun, B., Fall, K., Iannaccone, G., Knies, A., Manesh, M., Ratnasamy, S.: RouteBricks: exploiting parallelism to scale software routers. In: 22nd ACM Symposium on Operating Systems Principles (SOSP) (2009)

  10. DPDK Project: DPDK 16.11 Release Notes. http://dpdk.org/doc/guides/rel_notes/release_16_11.html (2016). Last visited 2016-03-27

  11. Emmerich, P., Gallenmüller, S., Raumer, D., Wohlfart, F., Carle, G.: MoonGen: A scriptable high-speed packet generator. In: 15th ACM SIGCOMM Conference on Internet Measurement (IMC’15) (2015)

  12. Emmerich, P., Raumer, D., Wohlfart, F., Carle, G.: A study of network stack latency for game servers. In: 13th Annual Workshop on Network and Systems Support for Games (NetGames’14). Nagoya, Japan (2014)

  13. Emmerich, P., Raumer, D., Wohlfart, F., Carle, G.: Performance characteristics of virtual switching. In: 2014 IEEE 3rd International Conference on Cloud Networking (CloudNet’14). Luxembourg (2014)

  14. ETSI: Network Functions Virtualisation (NFV); Architectural Framework, V1.1.1 (2013)

  15. Han, S., Jang, K., Panda, A., Palkar, S., Han, D., Ratnasamy, S.: Softnic: A software NIC to augment hardware. Technical Report UCB/EECS-2015-155, EECS Department, University of California, Berkeley (2015)

  16. He, Z., Liang, G.: Research and evaluation of network virtualization in cloud computing environment. In: Networking and Distributed Computing (ICNDC), pp. 40–44. IEEE (2012)

  17. Huggahalli, R., Iyer, R., Tetrick, S.: Direct cache access for high bandwidth network I/O. In: Proceedings of the 32nd Annual International Symposium on Computer Architecture, pp. 50–59 (2005)

  18. Hwang, J., Ramakrishnan, K.K., Wood, T.: Netvm: High performance and flexible networking using virtualization on commodity platforms. In: 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pp. 445–458. USENIX Association, Seattle (2014)

  19. Impressive Packet Processing Performance Enables Greater Workload Consolidation (2013)

  20. Intel DPDK vSwitch. https://01.org/sites/default/files/page/intel_dpdk_vswitch_performance_figures_0.10.0_0.pdf. Last visited 2016-03-27

  21. Intel DPDK vSwitch. https://github.com/01org/dpdk-ovs. Last visited 2016-03-27

  22. Intel DPDK: Data Plane Development Kit. http://dpdk.org. Last visited 2016-03-27

  23. Intel I/O Acceleration Technology. http://www.intel.com/content/www/us/en/wireless-network/accel-technology.html. Last visited 2016-03-27

  24. Jarschel, M., Oechsner, S., Schlosser, D., Pries, R., Goll, S., Tran-Gia, P.: Modeling and performance evaluation of an OpenFlow architecture. In: Proceedings of the 23rd International Teletraffic Congress. ITCP (2011)

  25. Kang, N., Liu, Z., Rexford, J., Walker, D.: Optimizing the “one big switch” abstraction in software-defined networks. In: Proceedings of the Ninth ACM Conference on Emerging Networking Experiments and Technologies, CoNEXT ’13, pp. 13–24. ACM, New York (2013). doi:10.1145/2535372.2535373

  26. Kevin, Traynor: OVS, DPDK and Software Dataplane Acceleration. https://fosdem.org/2016/schedule/event/ovs_dpdk/attachments/slides/1104/export/events/attachments/ovs_dpdk/slides/1104/ovs_dpdk_fosdem_16.pdf (2016). Last visited 2016-03-27

  27. Kohler, E., Morris, R., Chen, B., Jannotti, J., Kaashoek, M.F.: The click modular router. ACM Trans. Comput. Syst. (TOCS) 18(3), 263–297 (2000). doi:10.1145/354871.354874

    Article  Google Scholar 

  28. Larsen, S., Sarangam, P., Huggahalli, R., Kulkarni, S.: Architectural breakdown of end-to-end latency in a TCP/IP network. Int. J. Parallel Program. 37(6), 556–571 (2009)

    Article  MATH  Google Scholar 

  29. Martins, J., Ahmed, M., Raiciu, C., Olteanu, V., Honda, M., Bifulco, R., Huici, F.: ClickOS and the art of network function virtualization. In: 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14), pp. 459–473. USENIX Association, Seattle (2014)

  30. Meyer, T., Wohlfart, F., Raumer, D., Wolfinger, B., Carle, G.: Validated model-based prediction of multi-core software router performance. Praxis der Informationsverarbeitung und Kommunikation (PIK) (2014)

  31. Munch, B.: Hype Cycle for Networking and Communications. Report, Gartner (2013)

  32. Niu, Z., Xu, H., Tian, Y., Liu, L., Wang, P., Li, Z.: Benchmarking NFV software dataplanes. (2016). arXiv:1605.05843

  33. Open vSwitch. http://openvswitch.org. Last visited 2016-03-27

  34. OpenNebula. https://opennebula.org. Last visited 2016-03-27

  35. OpenStack. https://openstack.org. Last visited 2016-03-27

  36. OpenStack: Networking Guide: Deployment Scenarios. http://docs.openstack.org/liberty/networking-guide/deploy.html (2015). Last visited 2016-03-27

  37. Panda, A., Han, S., Jang, K., Walls, M., Ratnasamy, S., Shenker, S.: Netbricks: Taking the V out of NFV. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pp. 203–216. USENIX Association, GA (2016)

  38. Pettit, J., Gross, J., Pfaff, B., Casado, M., Crosby, S.: Virtual switching in an era of advanced edges. In: 2nd Workshop on Data Center Converged and Virtual Ethernet Switching (DC-CAVES) (2011)

  39. Pfaff, B., Pettit, J., Koponen, T., Amidon, K., Casado, M., Shenker, S.: Extending networking into the virtualization layer. In: Proceedings of workshop on Hot Topics in Networks (HotNets-VIII) (2009)

  40. Pfaff, B., Pettit, J., Koponen, T., Jackson, E., Zhou, A., Rajahalme, J., Gross, J., Wang, A., Stringer, J., Shelar, P., Amidon, K., Casado, M.: The design and implementation of open vswitch. In: 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15). USENIX Association (2015)

  41. Pongracz, G., Molnar, L., Kis, Z.L.: Removing roadblocks from SDN: OpenFlow software switch performance on Intel DPDK. In: Second European Workshop on Software Defined Networks (EWSDN’13) pp. 62–67 (2013)

  42. Ram, K.K., Cox, A.L., Chadha, M., Rixner, S.: Hyper-Switch: A scalable software virtual switching architecture. In: presented as part of the 2013 USENIX annual technical conference (USENIX ATC 13), pp. 13–24. USENIX, San Jose (2013)

  43. Rizzo, L., Carbone, M., Catalli, G.: Transparent acceleration of software packet forwarding using Netmap. In: INFOCOM, pp. 2471–2479. IEEE (2012)

  44. Rizzo, L.: Netmap: a novel framework for fast packet I/O. In: USENIX Annual Technical Conference (2012)

  45. Rizzo, L., Lettieri, G.: VALE, a switched ethernet for virtual machines. In: Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies (CoNEXT '12), pp. 61–72. ACM, New York (2012)

  46. Rotsos, C., Sarrar, N., Uhlig, S., Sherwood, R., Moore, A.W.: Oflops: an open framework for OpenFlow switch evaluation. In: Passive and Active Measurement, pp. 85–95. Springer (2012)

  47. Russell, R., Tsirkin, M.S., Huck, C., Moll, P. (eds.): Virtual I/O Device (VIRTIO) Version 1.0. OASIS Committee Specification 04 (2016)

  48. Salim, J.H., Olsson, R., Kuznetsov, A.: Beyond softnet. In: Proceedings of the 5th annual Linux Showcase & Conference, vol. 5, pp. 18–18 (2001)

  49. Tedesco, A., Ventre, G., Angrisani, L., Peluso, L.: Measurement of processing and queuing delays introduced by a software router in a single-hop network. In: IEEE Instrumentation and Measurement Technology Conference, pp. 1797–1802 (2005)

  50. Thomas, Monjalon: dropping librte_ivshmem. http://dpdk.org/ml/archives/dev/2016-June/040844.html (2016). Mailing list discussion

  51. Virtual Machine Device Queues: Technical White Paper (2008)

  52. Wang, G., Ng, T.E.: The impact of virtualization on network performance of amazon ec2 data center. In: INFOCOM, pp. 1–9. IEEE (2010)

  53. Whiteaker, J., Schneider, F., Teixeira, R.: Explaining packet delays under virtualization. ACM SIGCOMM Comput. Commun. Rev. 41(1), 38–44 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

This research has been supported by the DFG (German Research Foundation) as part of the MEMPHIS Project (CA 595/5-2) and in the framework of the CELTIC EUREKA Project SENDATE-PLANETS (Project ID C2015/3-1), partly funded by the German BMBF (Project ID 16KIS0460K). The authors alone are responsible for the content of the paper.

Author information

Authors and Affiliations

  1. Department of Informatics, Chair of Network Architectures and Services, Technical University of Munich, Boltzmannstr. 3, 85748, Garching, Germany

    Paul Emmerich, Daniel Raumer, Sebastian Gallenmüller, Florian Wohlfart & Georg Carle

Authors
  1. Paul Emmerich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Raumer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastian Gallenmüller
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Florian Wohlfart
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Georg Carle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sebastian Gallenmüller.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Emmerich, P., Raumer, D., Gallenmüller, S. et al. Throughput and Latency of Virtual Switching with Open vSwitch: A Quantitative Analysis. J Netw Syst Manage 26, 314–338 (2018). https://doi.org/10.1007/s10922-017-9417-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10922-017-9417-0

Keywords

Search

Navigation