A comprehensive survey on Carrier Ethernet Congestion Management mechanism

https://doi.org/10.1016/j.jnca.2014.09.003Get rights and content

Abstract

In the literature, congestion control schemes have been broadly studied. Nevertheless, this subject needs to be reviewed in the context of Carrier Ethernet. Congestion happens when the capacity cannot cover the resource demands. Nowadays, users are witnessing the networks saturation because of the expanding demands for bandwidth. The addressed issue is not only how to avoid congestion but also how to use all available capacity without overuse it or underuse it. To deal with congestion, link level algorithms drop or mark packets with increasing probability as buffer congestion increases. The most common solution is based on the transport layer algorithms to adjust the resources transmission rate by using these dropped or marked packets. Carrier Ethernet defined by layer two uses new proposals specified by the 802.1Qau standards committee in order to handle congestion problems. This paper surveys and studies different properties of Carrier Ethernet congestion control schemes. This paper also draws a parallel between the different schemes and point out the advantage and disadvantage of each one. Then, this paper presents a taxonomy of the Carrier Ethernet congestion control mechanism and correlate it with existing taxonomies.

Introduction

The original set of Ethernet LAN technology is popular due its capacity of affording the scaling and features needed for new generation carrier networks. The high bandwidth connectivity, the low-cost, and the ease of use are the most reasons behind the Ethernet network success. Originally designed as a technology for connecting computers in local area network (LAN), currently, Ethernet has also become the technology for metro (Raahemi et al., 2004), backbone (IEEE Computer Society, 2008, Nortel, 2007) and data center (O׳Hanlon, 2006, Cisco System Inc, 2008) networks. To cope with the fast evolving requirements for the rapidly growing Internet and increasing demand on bandwidth, Ethernet is evolving and the IEEE 802.3 standard group study technologies to realize 10 Gigabit Ethernet passive optical networks (10G E-PON) and 100 Gigabit Ethernet. An overview of Ethernet technologies evolution was given by Lam and Way (2008).

Carrier Ethernet augments the original set of Ethernet LAN technologies with support for new capabilities required to deliver services. The additional capabilities enable end users to build Metro and Wide Area Network, and service providers to build network infrastructure or deliver Ethernet based MAN or WAN telecommunication services. It provides flexible bandwidth increments and the ability to add new services using one technology (MEF5, 2004, MEF12, 2005). Therefore, operators tend to switch toward packet based Ethernet/IP technologies across their access and core networks in order to cost effectively support the rapidly escalating bandwidth requirements.

Today, there are exponential increase demands for applications like social networking, cloud computing (Ethernet Technology Summit and Exhibition, 2012, Buyya et al., 2008) and streaming video necessitate more powerful data centers (Kachris and Tomkos, 2012, Buyya et al., 2008). This makes a hard challenge to the data centers networking requiring efficient interconnection design with reduced latency and high bandwidth. Therefore, data center networks often uses Fiber Channel for high reliability. Kachris and Tomkos (2012) survey next generation data center networks using optical interconnects. Then, provide a taxonomy and comparison between the proposed schemes considering their main features such as scalability and connectivity. However, technology requiring Fiber Channel is more expensive than the Ethernet one. Hence, the IEEE 802.1 standards committee studied the issue of using Ethernet as the infrastructure to enable the Data Center applications (O׳Hanlon, 2006, Cisco System Inc, 2008).

Data Center traffic can be highly bursty, because when any single source needs full access to bandwidth to achieve the lowest latency in the absence of congestion, burst are injected without traffic shaping or policing. This is different environment from the Internet-type of networks, where aggregation of many low-speed flows (like FTP, web-browsing) determines the traffic characteristics. Lossless networks, like IfiniBand (InfiniBand Trade Association, 2004), Fibre channel, RapidIO (RapidIO Trade Association, 2011) and PCIe (PCI Express Specification) provide rapid response time, prevent buffer from saturation and prevent from throughput collapse caused by congestion. Congestion is then an important issue in the design of Data Center networks, as in any computer network, in order to create a lossless network. On the other hand, network that occasionally may lose some packets are called best effort or loosely. The well known are ATM networks and TCP/IP over Ethernet. TCP is designed to allow and relies on packet loss to detect congestion. Such solution is simple, cheap, and exports the problem from the core network to the edge nodes. However, packet drops recovery causes increased latency (Gusat et al., 2009).

Nevertheless, Ethernet networks, since it is appeared on 1973, run without congestion control in the data link layer. In order to eliminate long latencies caused by packet retransmission and reordering, Ethernet uses flow control at link level defined in the IEEE 802.3x that allows us to pause an Ethernet link. This technique is used in case of lack of sufficient resource buffer to receive the transmitting packet in the downstream switch. This can prevent from congestion collapse and reduce loss rate in the network, but also it may cause congestion spreading (Pfister and Norton, 1985) or tree saturation effect. Therefore a congestion control mechanism is a critical component in the network. In order to avoid drops due to congestion in the network, it is important for Ethernet used in data center or metro networks to support congestion management. Consequently, IEEE 802.1 standards committee is developing new proposals for congestion signaling in Carrier Ethernet networks.

In an effort to keep up with demand and maximize QoS, IEEE802.1Qau (IEEE, 2010) is deploying congestion control mechanism for Carrier Ethernet. The standard specified congestion management mechanisms enabling switches to notify congestion information to edge stations with the functionalities of adjusting their transmission rate to reduce frame loss.

Depending on network topology each packet usually routed over a certain number of intermediate nodes (switches). According to IEEE802.1 Qau, the intermediate switches typically have a queue that grows when congestion occur, then frames are dropped when the queue length exceeds a defined threshold. In addition, flows can be managed by both sender and intermediate nodes. Furthermore, congestion control measurement can be proceeded by intermediate nodes or/and by the receiver (destination). Congestion information, called also feedback or congestion notification, can be reflected back to the reaction point by the intermediate node (congestion point) or by the destination node. These entities are then called reflection points. Depending on the location of the reflection point, three congestion control varieties can be defined as follows:

  • Reflection point resides at the intermediate node (switch): It consists of Backward notification Congestion Management (BCM), the advantage of this mechanism is to deal with a sudden increases in load faster. Ethernet Congestion Management (ECM) (Lu et al., 2007) and Quantized Congestion Notification (QCN) (Alizadeh et al., 2008) are such mechanisms.

  • Reflection point resides at the destination node: It consists of Forward notification Congestion Management (FCM). This mechanism outperforms BCM in fairness and stability. Forward Explicit Congestion Notification (FECN) (Jiang et al., 2007) is such mechanisms.

  • Reflection point resides at the intermediate node (switch) and the destination node: It consists of a combination between the BCM and the FCM mechanism. Consequently, we call it the Hybrid Notification Congestion Management mechanisms. Hybrid Notification Congestion Management mechanisms are: 3 point QCN (3ptQCN), Extended Ethernet Congestion Management (E2CM) (Gusat et al., 2007) and Enhanced Forward Explicit Congestion Notification (E-FECN) (So-In et al., 2008).

These Ethernet congestion control mechanisms are the main focus of the present study in this paper.

The congestion control mechanisms should be able to provide some performance features.

  • Fairness: is based on Max–Min fairness (Bonald et al., 2006) which considers fairness in sharing a scarce queue׳s space and gives each connection an equal part. The Max–Min fairness approach provides fairness by equally sharing the bottleneck. Ethernet congestion control proposed mechanisms allow equal throughput rate to all flows involved in the bottleneck path. Therefore, mechanisms׳ fairness results in sharing only the resources between the source and the most congested intermediate node.

  • Prevent feedback implosion: A general drawback mentioned with respect to feedback messages is the additional traffic in the reverse direction on what it may be a congested path. Nevertheless, the need of congestion state information for senders, in order to be responsive, has to be taking into account. Therefore some proposed mechanism sends feedback message within probability value (like FECN, E2CM). Others try to reduce the feedback generation by getting rid of positive feedback from the congestion point (like QCN).

  • Stability: Switch queue occupancy processes should not fluctuate, causing underutilization of the resource or overutilization leading to frame drops. However, queue size should be controlled to be maintained at a target threshold length. This is crucial when trying to control a shallow queue.

  • Responsiveness: Ethernet link throughput can vary with time due to flow fluctuation or the appearance of bottlenecks or the arrival of new sources, etc. the algorithm needs to adjust source rates according to these variations.

  • Simple to implement: The algorithm should be simple in order to be implemented in hardware. Complicated rates computing, parameters tuning and control loop gains should be omitted.

The main problem to be solved is to achieve lossless flow transport in Carrier Ethernet. In order to resolve this problem Ethernet protocol should be enhanced to support the new Carrier Ethernet services. Congestion control algorithms are a key component of data transport in Carrier Ethernet. Efficient congestion control schemes for Carrier Ethernet should be implemented to avoid congestion and to get lossless transport traffic. In fact, the sources need to be notified to adjust their sending rate according the status of the network in order to manage congestion and avoid traffic loss. In this context, this paper surveys, classifies and compares the existing schemes for congestion control implemented for Carrier Ethernet.

The contributions of this paper are as follows: (a) the paper presents a survey of existing closed loop flow control techniques used in the literature and identifies the advantage and disadvantage of these techniques. (b) The paper studies the Carrier Ethernet technologies and services. (c) The paper elaborates a taxonomy of the Carrier Ethernet congestion control mechanism and correlate it with existing taxonomies in literature. (d) The paper presents a comparative study of different approaches of congestion control for Carrier Ethernet in satisfying the performance features in MAN, WAN and data center networks context.

The rest of this paper is organized as follows. Section 2 surveys some consideration for congestion notification and control in the literature. Then, Section 3 shows Carrier Ethernet services and technologies to comply with the specific requirements of several application fields. Next, Section 4 studies the different Ethernet Congestion Management schemes: Backward Congestion Management mechanisms, Forward Congestion Management mechanisms and Hybrid congestion Management mechanisms. Finally, the summary of the paper is provided in Section 5.

Section snippets

Background

Congestion management in high-speed networks stills a debate topic. A paradoxically belief exists on most issues. Jain (1992) demystifies different issues for congestion management and point out weaknesses in several of the ideas that have been proposed. Thus, the author expound a number of old and new myths about congestion. Congestion state leads to performance degradation due to network resource saturation. Congestion can occur when there are temporary failures in the network. In a network

Carrier Ethernet network overview

The conception of Next Generation Network (NGN) carries out a new infrastructure with new innovation and competences. An NGN architecture is built with the intention of incorporating variant services like data, voice and video, and smooth convergence of mobile and fixed networks (ETSI, 2006, Knightson et al., 2005).

Ethernet is well known as scalable data networking solution in LAN environments, inexpensive, high bandwidth connectivity and ease of use. In addition, the continued grade of the

Ethernet Congestion Management schemes

TCP is the broadly adopted technique for congestion management. Some research works have been elaborated to answer the question of how does TCP congestion control performs over data center network supported by Carrier Ethernet. Crisan et al. (2011) find that TCP Cubic allows aggressively and slows convergence of congestion windows. On the other hand, TCP New Reno requires more parametrical retuning than Vegas. Whether RTT independence, as in TCP Cubic and TCP BIC, is performance deleterious in

Related work to survey and overview Carrier Ethernet technologies

This section presents related works that survey and overview Carrier Ethernet technologies by highlighting their main contributions. There are many research works that discussed requirements, services, design, standard protocols, applications, technologies for Carrier Ethernet. However, as far as we know, there is no survey or investigation focused on over-viewing congestion management schemes proposed solutions for Carrier Ethernet. We bring out that congestion management schemes that proposed

Summary

Carrier Ethernet is not about the Ethernet within LANs that we are used to know. Therefore, this paper recaps Carrier Ethernet services and technologies. Carrier Ethernet defined by layer two uses new proposals specified by the 802.1Qau standards committee in order to handle congestion problems. The backward congestion control, the forward congestion control and hybrid congestion control are three proposal schemes for congestion notification in data center Ethernet networks and WAN under the

References (143)

  • D.M. Chiu et al.

    Analysis of the increase and decrease algorithms for congestion avoidance in computer networks

    J Comput Netw ISDN Syst

    (1989)
  • K.W. Fendick et al.

    Analysis of a rate-based feedback control strategy for long haul data transport

    ELSEVIER Perform Eval

    (1992)
  • A. Afanasyev et al.

    Host-to-host congestion control for TCP

    IEEE Commun Surv Tutor

    (2010)
  • Alizaddeh M, Atikoglu B, Kabbani A, Lakshmikantha A, Pan R, Prabhakar B, et al. Data center transport mechanisms:...
  • Alizadeh M, Atikoglu B, Kabbani A, Lakshmikantha A, Pan R. Data center transport mechanisms: congestion control theory...
  • Alizadeh M, Kabbani A, Atikoglu B, Prabhakar B. Stability analysis of QCN: the averaging principle. In: ACM SIGMETRICS;...
  • D. Allan et al.

    Shortest path bridgingefficient control of larger ethernet

    IEEE Commun Mag

    (2010)
  • G. Appenzeller et al.

    Sizing router buffers

    ACM SIGCOMM Comput Commun

    (2004)
  • Armbrust M, Fox A, Grifth R, Joseph AD, Katz R, Konwinski A, Lee G, et al. Above the clouds: a Berkeley view of cloud...
  • S. Athuraliya et al.

    REMactive queue management

    IEEE Netw

    (2001)
  • Atikoglu B, Kabbani A, Pan R, Prabhakar B. QCN: first benchmark simulation....
  • Atikoglu B, Kabbani A, Pan R, Prbhakar B, Seaman M. The origin, evolution and current status of QCN....
  • Bergamasco D. CN-SIM: a baseline simulation scenario. In: IEEE 802.1 meeting;...
  • Bergamasco D. Ethernet congestion manager (ECM). In: IEEE802 plenary meeting; March 13,...
  • Berger L, Fedyk D. Generalized MPLS (GMPLS) support for metro ethernet forum and g.8011 user network interface (UNI)....
  • A. Berger et al.

    Dimensioning bandwidth for elastic traffic in high-speed data networks

    IEEE/ACM Trans Netw

    (2000)
  • Bermasco D. Data center Ethernet congestion management: backward congestion notification. In: IEEE 802.1 meeting;...
  • S. Bidkar et al.

    On the design, implementation, analysis and prototyping of a 1-microsecond, energy-efficient, carrier-class optical-ethernet switch router

    IEEE J Lightw Technol

    (2014)
  • T. Bonald et al.

    A queue analysis of max–min fairness, proportional fairness and balanced fairness

    Queueing Syst: Theory Appl

    (2006)
  • Bottorff, P. 2006. Draft standard for local and metropolitan area networks–virtual bridged local area...
  • Braden B, Clarck D, Crowcroft J, Davie B, Deering S, Estrin D, et al. Recommendations on queue management and...
  • Brodcom connection everything. BCN simulation results 100 flows scenario; November 2,...
  • Bullot H, Les Cottrell R. Evaluation of advanced TCP stacks on fast long-distance production networks. J Grid Comput...
  • Buyya R, Yeo CS, Venugopal S. Market-oriented cloud computing: vision, hype, and reality for delivering it services as...
  • Charny A, Clarck DD, Jain R. Congestion control with explicit rate indication. In: IEEE ICC׳95 Seattle, gateway to...
  • D. Chen et al.

    PI/PID controller design based on direct synthesis and disturbance rejection

    Ind Eng Chem Res

    (2002)
  • CISCO public information. New peering standards for Ethernet exchanges: simplify interconnections and enable new...
  • Cisco SystemInc. Cisco metro Ethernet access services;...
  • Cisco System Inc. Data Center Ethernet (DCE). www.cisco.com/go/learnetspace;...
  • Cisco System Inc. Configuring IEEE standard-compliant Ethernet CFM in a service provider network; February...
  • Cisco System Inc. ITU-T Y.1731 performance monitoring in a service provider network; March...
  • Cisco System. Inc. Carrier Ethernet configuration guide, Cisco IOS release 15.2S;...
  • Cisco Systems. Metro Ethernet WAN services and architectures;...
  • D.J. Cooper

    Practical process control using LOOP-PRO software

    (2005)
  • S. Cornay et al.

    Carrier ethernet basics

    (2011)
  • Crisan D, Anghel AS, Brike R, Minkenberg C, Gusat M. Short and fat: TCP performance in CEE datacenter networks. In:...
  • G. de Veciana et al.

    Stability and performance analysis of networks supporting elastic services

    IEEE ACM Trans Netw

    (2001)
  • Devkota P, Reddy ALN. Performance of quantized congestion notification in TCP incast scenarios of data centers. In:...
  • S.I. Eludiora et al.

    Performance analysis of metro ethernet network (MEN) services over asynchronous transfer mode (ATM) services for enterprise

    IJCSI Int J Comput Sci Iss

    (2011)
  • Ethernet Technology Summit and Exhibition. Featuring 40/100GbE, convergence, cloud computing, virtualization. Ethernet...
  • ETSI. Telecommunications and Internet converged services and protocols for advanced networking NGN release 1, release...
  • K. Fall et al.

    Simulation-based comparisons of tahoe, reno and SACK TCP

    ACM SIGCOMM Comput Commun Rev

    (1996)
  • Floyd S. High speed TCP for large congestion windows. 〈http://tools.ietf.org/pdf/rfc3649.pdf〉; December...
  • S. Floyd et al.

    Promoting the use of end-to-end congestion control in the internet

    IEEE/ACM Trans Netw

    (1999)
  • S.B. Fredj et al.

    Statistical bandwidth sharinga study of congestion at flow level

    ACM SIGCOMM Comput Commun Rev

    (2001)
  • Ghaemi RV. Ethernet OAM test applications. JDS Uniphase Corporation; January...
  • Gusat M, Minkenberg C, Luijten R. Extended Ethernet congestion management (E2CM): per path ECM—a hybrid proposal. IBM...
  • Gusat M, Minkenberg C, Pajack GJ. Flow and congestion control for data center networks. IBM research report. Computer...
  • Hashem ES. Analysis of random drop for gateway congestion control. In: Report LCS TR-465. Cambridge, MA: MIT Laboratory...
  • Hayashi Y, Itsumi H, Yamamoto M. Improving fairness of quantized congestion notification for data center ethernet...
  • Cited by (15)

    • Survey on link layer congestion management of lossless switching fabric

      2018, Computer Standards and Interfaces
      Citation Excerpt :

      The special requirement of no packet loss makes the loss-based congestion control algorithm useable at the link layer. Although the existing congestion control algorithms have already been summarized in [13], [14], they focus on the detailed mechanisms instead of the way to guarantee no packet loss. In contrast, we focus on the congestion management, including both the way of guarantee no packet loss and the congestion control algorithms which cooperates with the mechanism for losslessness.

    • FSQCN: Fast and simple quantized congestion notification in data center ethernet

      2017, Journal of Network and Computer Applications
      Citation Excerpt :

      With the development of large scale data centers, networking technology is expected to avoid packet loss and provide low latency (Zhang et al., 2015, 2016; Mliki et al., 2015).

    • Conceptual Model of Perspective Carrier Ethernet Transport Communication Network Control System for Public Joint Stock Company «Russian Railways»

      2022, 2022 Systems of Signals Generating and Processing in the Field of on Board Communications, SOSG 2022 - Conference Proceedings
    • An Ethernet Mapping High-Level Data Link Control Circuit Design

      2022, Lecture Notes on Data Engineering and Communications Technologies
    • Evolving Fast Innovation in Next-Generation Networking through Flexible and Customized Softwarization and Slicing Capabilities

      2020, 2020 IEEE Conference on Network Function Virtualization and Software Defined Networks, NFV-SDN 2020 - Proceedings
    View all citing articles on Scopus
    View full text