Efficient measurement of round-trip link delays in software-defined networks
Introduction
Link delays, including one-way link delays and round-trip link delays, are important metrics for network performance assessment and troubleshooting. Consequently, the link delay measurement has received a lot of research interests from both academical and industrial communities (Goel et al., 2016; Guo et al., 2015; Shahzad and Liu, 2014; Wang et al., 2016; Lu et al., 2009; Lee et al., 2010) in recent years. Accurately measuring one-way link delays is surprisingly hard. Xia and Tse (2006) have been proved that not all one-way link delays are identifiable unless every link directly connects two monitors on the end nodes. Evidently, the cost of accurately measuring one-way link delays is very high. Thus, to monitor the network performance with a low cost, network operators prefer to measure round-trip link delays. In practical networks, round-trip link delays are also very useful for network operators. With the round-trip link delays, network operators can quickly located the network problem to some bidirectional links (Tan et al., 2019), and they even can exactly locate the problematic one-way links by incorporating other performance metrics (e.g., link load). Therefore, we also study the round-trip delay measurement problem in this paper.
The existing link delay measurement approaches can be broadly divided into two categories: internal and external approaches. Internal approaches (Shahzad and Liu, 2014; Wang et al., 2016; Lu et al., 2009; Lee et al., 2010) measure link delays using the built-in measurement modules in network devices (e.g., routers and switches). The internal approaches can accurately measure fine-grained delay of links and flows, however, they have the following drawbacks: First, most network devices do not support delay measurement due to lack of specialized delay measurement modules; Second, measuring link delays at each node and collecting statistics from all node are not scalable in large networks.
In contrast, external approaches (Guo et al., 2015; Gopalan and Ramasubramanian, 2012; Wang et al., 2015; Gopalan and Ramasubramanian, 2014; Ma et al., 2013a,b, 2014; Duffield and Presti, 2000) infer link delays from external measurements, which can be obtained by monitors connecting to a subset of network nodes. Comparing with internal approaches, external approaches are much easier to implement and incur lower overhead. The external approaches can be further classified into hop-by-hop and end-to-end approaches. Hop-by-hop approaches leverage probing tools such as ping, traceroute, and pathchar to measure the delays of hop-by-hop links on the probing paths. These tools probe link delays by exchanging Internet Control Message Protocol (ICMP) packets among the nodes on the probing paths. However, for security concerns, the ICMP may be disabled in some nodes, which makes these tools infeasible to use in real networks. Furthermore, per-hop measurement incurs high extra traffic load.
To overcome the shortcomings of hop-by-hop approaches, the end-to-end approaches are proposed to measure link delays. End-to-end approaches first measure the accumulated end-to-end delays of a set of selected paths, and then use the network tomography techniques (Lawrence and Michailidis, 2006) to infer the delays of individual links from the measured delays of end-to-end paths. End-to-end approaches do not require special measurement modules and protocols to be run at each node, and they also consume less resources (e.g., bandwidth and monitors) than hop-by-hop approaches. Thus, in this paper, we also use the end-to-end approach to measure round-trip link delays.
In order to get the end-to-end delays, end-to-end approaches require to place some monitors in a network and construct some measurement paths starting and ending at the monitors. Given the vector of measured end-to-end delays and the binary routing matrix for the measurement paths, the problem of measuring round-trip link delays using end-to-end approaches can be represented as a system of linear equations (Gopalan and Ramasubramanian, 2012; Ma et al., 2013b). In the theory of linear algebra, the linear system has a unique solution if and only if the binary routing matrix is invertible, i.e., the number of linear independent measurement paths must equal to the number of unknown round-trip link delays (denoted by m). To uniquely identify round-trip link delays, we need to construct m linear independent measurement paths starting and ending at monitors. Conversely, if the number of constructed measurement paths is less than m, the round-trip link delay measurement problem becomes an under-determined problem. In this case, we can use the statistical technique to get estimated round-trip delays based on the given measurements. However, the estimated results may have high errors. In this paper, our goal is to uniquely identify round-trip link delays.
To uniquely identify round-trip link delays, we mainly face the following two challenges in today's IP networks. First, since most IP networks use the shortest path routing protocol and do not support explicit routing, the measurement paths cannot be implemented in these networks. Second, with the shortest-path routing constraint, a large number of monitors may be required to uniquely identify round-trip delays of all links (Wang et al., 2015), which makes round-trip link delay measurement costly and even infeasible(Wang et al., 2015).
On the other hand, the emergence of SDN (Masoudi and Ghaffari, 2016; Feng et al., 2017; Yu et al., 2017) paves the way for efficiently handling the above challenges of measuring round-trip link delays in traditional IP networks. SDN separates the control plane and the data plane, and the control plane, running on a logically centralized network controller, can control the behavior of switches in data plane through standard protocols (e.g., OpenFlow) (ONF, 2012). In SDN networks, we can easily set up various measurement paths (simple paths, cyclic paths, and multicast trees) by leveraging the flexible forwarding control capability of SDN. Recently, the link delay or end-to-end delay measurement problem in SDN networks also attracts some research efforts (Van Adrichem et al., 2014; Phemius and Bouet, 2013; Shibuya et al., 2014; Atary and BremlerBarr, 2016; Li et al., 2018; Yu et al., 2015; Liao et al., 2018) (see Section 2). These studies demonstrate that SDN networks have potential capability to enable low-cost and accurate link delay measurement. However, to save measurement resources (monitor, TCAM and bandwidth) in SDN networks, we also need to consider how to design efficient measurement scheme, including measurement path construction and flow rule design for realizing measurement paths, how to place monitors, and how to assign links to be measured by each monitor under the processing capacity constraint of monitors.
In this paper, we investigate the round-trip link delay measurement problem in SDN networks. Specifically, we design a measurement scheme for measuring round-trip link delays in SDN networks and address the monitor placement and link assignment problem under the monitor capacity constraint. To reduce the measurement cost, we aim to minimize the number of required monitors and TCAM entries as well as the bandwidth consumed by probe packets. Our contributions are summarized as follows.
- ā¢
We design a low-cost round-trip link delay measurement scheme, which leverages the flexible packet forwarding control capability of SDN networks to facilitate the round-trip link delay measurement process. Specifically, the scheme constructs measurement paths for probe packets and design flow rules to realize the constructed measurement paths.
- ā¢
We address the Monitor Placement and Link Assignment (MPLA) problem under the monitor capacity constraint. We first formulate the problem as a Mixed Integer Linear Programming (MILP) problem, then we prove that the problem is NP-hard, and at last, we propose an efficient heuristic algorithm called MPLA Algorithm based on Biding Strategy (MPLAA-BS) to solve the problem.
- ā¢
We conduct extensive simulations to evaluate the performance of the proposed round-trip link delay measurement scheme and MPLAA-BS. The simulation results show that the proposed round-trip link delay measurement scheme can accurately measure round-trip link delays with low cost, and the MPLAA-BS can find near-optimal solutions for the MPLA problem.
The remainder of the paper is organized as follows. The related works on link delay measurement are reviewed in Section 2. Section 3 describes the network model and motivations of this work. Section 4 presents the round-trip link delay measurement scheme. Section 5 gives the MILP formulation for the monitor placement and link assignment problem and proves that the problem is NP-hard. The heuristic algorithm for the monitor placement and link assignment problem are presented in Section 6. Performance evaluations are shown in Section 7, and Section 8 concludes this paper.
Section snippets
Related work
Since external approaches are much easier to implement and incur much lower overhead than internal approaches in production networks, how to accurately infer link delays from external measurements in traditional IP networks has attracted significant research interests (Gopalan and Ramasubramanian, 2012; Ma et al., 2013a,b, 2014; Duffield and Presti, 2000; Shih and Hero, 2001; Xia and Tse, 2006; Presti et al., 2002; Gopalan and Ramasubramanian, 2014; Wang et al., 2015; Gurewitz and Sidi, 2001;
Network model and motivations
We model an SDN network as a connected directed graph G(V, E), where the graph nodes, V, are the SDN switches and the edges, E, are the links between SDN switches. Let nāÆ=āÆ|V | and māÆ=āÆ|E| denote the number of nodes and links, respectively. Each directed link (i, j)āÆāāÆE has a one-way delay dij, which may vary over time. The Round-Trip Delay (RTD) of link (i, j) equals to dijĀ +Ā dji. In real networks, link delays may fluctuate frequently over time (e.g., vary for every millisecond (Zhang et al.,
The round-trip link delay measurement scheme in SDN networks
Comparing with traditional IP networks, SDN networks have much higher flexibility in the aspect of packet forwarding control. For example, SDN networks can easily realize multicast forwarding by adding multiple actions in a flow rule, and SDN networks also allow packets to be forwarded on paths with loops. In this section, we will design an efficient round-trip link delay measurement scheme by leveraging the packet forwarding flexibility of SDN. The scheme mainly involves two steps:
Problem formulation
In this section, we first introduce the monitor placement and link assignment problem, then we prove the problem is NP-hard, and at last, we formulate the problem as a Mixed Integer Linear Programming (MILP) problem.
The heuristic algorithm
Since the MPLA problem is NP-hard, we propose a heuristic algorithm called MPLA Algorithm based on Biding Strategy (MPLAA-BS) to solve the problem. The MPLAA-BS is inspired by the primal-dual algorithm designed for solving the facility location problem (Jain et al., 2002). The key idea of MPLAA-BS is to make monitor placement and link assignment decisions based on the price offered by each link that needs to be measured.INPUT: An SDN
Performance evaluation
To evaluate the performance of the proposed round-trip link delay measurement scheme and MPLAA-BS, we conducted extensive simulations on real topologies, including the GĆANT topology (GEANT, 2012) and some topologies taken from Topology Zoo (2013). All the topologies used in the simulations are listed in TableĀ 4. The MPLAA-BS is implemented with Python 3.6.5, and the round-trip link delay measurement scheme is implemented and tested on Mininet.
The round-trip delay of a link (u, v) consists of
Summary and future work
In this work, we designed an efficient round-trip link delay measurement scheme for SDN networks, and to minimize measurement cost (i.e., the total cost of monitors, flow rules, and bandwidth) and fulfill monitor capacity constraint, we addressed the monitor placement and link assignment problem, whose objective is to minimize measurement cost by optimizing the strategies for the monitor placement and link assignment. For the monitor placement and link assignment problem, we first formulated
Declaration of competing interest
We declare that we have no conflict of interest.
Acknowledgment
We would like to thank the reviewers for their valuable comments. This work is partially supported by National Key R&D Program of China (2018YFB2100100, 2018YFC0831002), National Natural Science Foundation of China (61671130, 61701058, 61271165) and the project on Public Safety Risk Control and Emergency Technical Equipment(2018YFC0831002).
Xiong Wang is an associate professor in school of information and communication at the University of Electronic and Science of China, Chengdu, China. He received his Ph.D. in communication and information system from the University of Electronic and Science of China, Chengdu, china, in 2008. From 2013 to 2014, he was a visiting scholar in electrical and computer engineering at the University of California, Davis, CA, USA. His research interests include network measurement, modeling and
References (46)
- et al.
One-way queuing delay measurement and its application on detecting ddos attack
- et al.
Software defined networks: a survey
- et al.
007: democratically finding the cause of packet drops
- et al.
Efficient round-trip time monitoring in openflow networks
- et al.
Measuring and characterizing end-to-end route dynamics in the presence of load balancing
- et al.
Multicast inference of packet delay variance at interior network links
- et al.
Locator/identifier split networking: a promising future internet architecture
- et al.
Link delay estimation via expander graphs
Geant
- et al.
Survey of end-to-end mobile network measurement testbeds, tools, and services
On identifying additive link metrics using linearly independent cycles and paths
On the maximum number of linearly independent cycles and paths in a network
Pingmesh: a large-scale system for data center network latency measurement and analysis
Estimating one-way delays from cyclic-path delay measurements
Network emulation with netem
Statistical characteristics of active ip one way delay measurements
A new greedy approach for facility location problems
Network tomography: a review and recent developments
Not all microseconds are equal: fine-grained per-flow measurements with reference latency interpolation
Llmp: exploiting lldp for latency measurement in software-defined data center networks
An efficient and accurate link latency monitoring method for low-latency software-defined networks
Identifiability of link metrics based on end-to-end path measurements
Monitor placement for maximal identifiability in network tomography
Cited by (4)
Flowlet-level multipath routing based on graph neural network in OpenFlow-based SDN
2022, Future Generation Computer SystemsApplication Layer Multicast Model with Low Delay and High Stability
2023, Journal of Donghua University (English Edition)Enabling fast path delay measurement using closeness centrality
2022, Internet Technology LettersAdopting MQTT for a multi protocols IoMT system
2022, International Journal of Electrical and Computer Engineering
Xiong Wang is an associate professor in school of information and communication at the University of Electronic and Science of China, Chengdu, China. He received his Ph.D. in communication and information system from the University of Electronic and Science of China, Chengdu, china, in 2008. From 2013 to 2014, he was a visiting scholar in electrical and computer engineering at the University of California, Davis, CA, USA. His research interests include network measurement, modeling and optimization, algorithm analysis and design, network management in communication networks.
Yukun Yang is a graduate student in the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China. He will get her master degree in June 2020. His research interests are software defined networking and network measurement.
Hanyu Liu She received her master degree in communication and information system from the University of Electronic and Science of China. She is an engineering in Huawei Technologies Co. Ltd. Her research interests are software defined networking and network measurement.
Jing Ren received her Ph.D. degree in communication and information system from the University of Electronic Science and Technology of China, Chengdu, China, in 2007. She now is a lecturer in school of information and communication at the University of Electronic and Science of China, Chengdu, China. Her research interests include network architecture and protocol design, information-centric networking, and softwaredefined networking.
Sheng Wang received the B.S. degree in electronic engineering, and the M.S. and Ph.D. degrees in communication engineering from the University of Electronic Science and Technology of China, Chengdu, China, in 1992, 1995, and 2000, respectively. He now is a professor in the school of information and communication at the University of Electronic Science and Technology of China, Chengdu, China. His research interests include planning and optimization of wire and wireless networks, next generation of internet, and next-generation optical networks.
Shizhong Xu received his B.S., M.S., and Ph.D. degrees in electrical engineering from the University of Electronic Science and Technology of China, Chengdu, China, in 1994, 1997, and 2000, respectively. He now is a professor in the school of information and communication at the University of Electronic Science and Technology of China, Chengdu, China. His research interests include Internet of Things, next generation network and network science.
Shui Yu is currently a full Professor of School of Software, University of Technology Sydney, Australia. Dr Yu's research interest includes Security and Privacy, Networking, Big Data, and Mathematical Modelling. He has published two monographs and edited two books, more than 200 technical papers, including top journals and top conferences, such as IEEE TPDS, TC, TIFS, TMC, TKDE, TETC, ToN, and INFOCOM. Dr Yu initiated the research field of networking for big data in 2013. His h-index is 33. Dr Yu actively serves his research communities in various roles. He is currently serving the editorial boards of IEEE Communications Surveys and Tutorials, IEEE Communications Magazine, IEEE Internet of Things Journal, IEEE Communications Letters, IEEE Access, and IEEE Transactions on Computational Social Systems. He has served many international conferences as a member of organizing committee, such as publication chair for IEEE Globecom 2015, IEEE INFOCOM 2016 and 2017, TPC chair for IEEE BigDataService 2015, and general chair for ACSW 2017. Dr Yu is a final voting member for a few NSF China programs in 2017. He is a Senior Member of IEEE, a member of AAAS and ACM, the Vice Chair of Technical Commuittee on Big Data of IEEE Communication Society, and a Distinguished Lecturer of IEEE Communication Society.