A scalable method for DCLC problem using hierarchical MDP model☆
Introduction
The continuous growth in network applications with various QoS requirements is calling for better transmission qualities than the best effort service. These emerging applications (video, audio, interactive multimedia, teleconferencing, etc.) have stringent QoS requirements and pose a particular difficulty for the QoS routing. For example, the delay-sensitive applications such as real-time voice and video require the data stream to be received at the destination within a certain time. Generally, the QoS requirements are represented by constraints imposed upon the corresponding performance metrics such as delay, jitter, cost, etc. The end-to-end delay constraint is the most prominent factor of these constraints for QoS support. In addition, the traffics generally utilize a significant amount of resources usually measured by the cost metric. Hence the need for QoS routing is the ability to satisfy the QoS requirements of the traffics and to better optimize resource utilization. In this paper, we focus on the most typical delay-sensitive routing problem which has been extensively studied in the past decade, the delay-constrained least cost (DCLC) routing problem[1], i.e., to find a path that has the minimal cost subject to a delay constraint.
The DCLC routing plays a very important role in the field of QoS routing. Much work has been done to solve the DCLC problem. However, the existing DCLC routing algorithms still have one or more distinct drawbacks, such as high computational complexity, high communication complexity and low success ratio to obtain the optimal path, that seriously hinder the application of DCLC routing to the practical network environment. Moreover, DCLC routing has to consider not only the application to a specified network size, but also the application to a larger network scale for the scalability requirement. Therefore, we see the DCLC problem from a completely different angle to find a suitable solution that can achieve 100% success ratio to obtain the optimal path with low computational complexity, low communication complexity and better scalability.
With the continuing growth of the network size, the existing heuristic algorithms for DCLC routing will not work well, leading to what is called the scalability problem. So far, the scalability problem of DCLC routing has seldom been addressed. Therefore our focus is to propose a feasible solution for resolving the scalability problem of DCLC routing. We notice that the frequent advertisement of routing information is the critical factor for the scalability problem of DCLC routing. So our purpose is centered on the decrease of the advertisement. Under the steady network and link independence assumptions, MDP theory is appropriately employed to compute an optimal path for the DCLC problem by means of link by link way, which is not necessary to frequent advertise the routing information. To achieve this purpose, we rebuild a two-level MDP model for DCLC routing, which is an effective structure of offering better scalability. For the lower level, we adopt the finite horizon cost MDP to determine the links of a path within a domain. For the upper level, we adopt the infinite-horizon discounted cost MDP to determine the inter-domain links. Thus, the end-to-end DCLC routing is determined regardless of the network size and then the scalability problem is resolved.
This paper is organized as follows. We start with the related works about the solutions for the DCLC problem in Section 2 and present a detailed description of our proposed hierarchical MDP model for DCLC routing in Section 3. In Section 4, we give the method of solving the optimal policy for the proposed MDP model. We then analyze the performance via simulations in Section 5. We conclude this paper in Section 6.
Section snippets
Related works
In this section we give a brief review of the different routing algorithms for the DCLC problem proposed in previous literatures. The DCLC routing has been formulated in [1]. It is a cost optimization problem subjected to the delay constraint. Due to the NP-complete property, the DCLC routing cannot be solved in polynomial time. Heuristic algorithms have been proposed to tackle this problem. The earliest work is accredited to Hassin’s two ε optimal approximation algorithms with the costs less
The proposed hierarchical MDP model for DCLC routing
Generally, the scalability problem in current DCLC routing algorithms may derive from two aspects. First, the amount of information used to describe the network state may increase dramatically with the increasing expansion of network scale. Second, the deterministic network information is used to compute a feasible path. It is these two factors that make the transmission and storage of network state information more difficult. So it becomes impossible to maintain up-to-date network state
A simple example
For the illustration of solving the optimal policy, we give a simple example depicted by Table 1. In order to simplify the solving process for explicitly elaborating the details of the link selection, the tandem topology is considered to be used in the example.
Let the state space be . denotes that the link is more congested than the former link. denotes that the link is less congested than the former link. The action space is denoted by . denotes that the link with state
Performance simulation
In order to evaluate the performance of the proposed method, Hierarchically Parallel Time Variation (HPTV) DCLC, extensive simulations have been used to compare the performance of HPTV with LARRC and DCUR. The simulations run on OPNET Modeler. The tandem multi-domain network is used to provide a larger network scale. For the simulation we built 5 random domains with 4 average traversed nodes per domain. The Waxman’s method [19] is employed to generate each domain topology. For each domain, an
Conclusion
In this paper, we propose a scalable MDP-based routing algorithm for the DCLC problem. This algorithm is based on our proposed hierarchical MDP model in which the infinite-horizon discounted cost model is used in the upper level for the inter-domain link selection. Since the infinite-horizon discounted cost model is independent of network scale, the scalability problem is resolved. We give the detailed modeling process and give the method for solving the optimality policy. A simple example
References (21)
- et al.
An efficient quality of service routing algorithm for delay-sensitive applications
Computer Networks
(2005) Approximation schemes for the restricted shortest path problem
Mathematics of Operations Research
(1992)- R. Widyono, The design and evaluation of routing algorithms for realtime channels, Technical Report TR-94-024,...
- et al.
Heuristic methods for delay constrained least cost routing using k-shortest-paths
IEEE Transactions on Automatic Control
(2006) - et al.
Achieving 100% Success Ratio in Finding the Delay Constrained Least Cost Path
(2004) - et al.
Enhancing -approximation algorithms with the optimal linear scaling factor
IEEE Transactions on Communications
(2006) - et al.
A distributed algorithm for delay constrained unicast routing
IEEE/ACM Transactions on Networking
(2000) - et al.
The Enhanced Ticket-Based Routing Algorithm
(2002) - et al.
Estimated Link Selection for DCLC Problem
(2004) - et al.
Lagrange Relaxation Based Method for the QoS Routing Problem
(2001)
Cited by (1)
Function Split between Delay-Constrained Routing and Resource Allocation for Centrally Managed QoS in Industrial Networks
2016, IEEE Transactions on Industrial Informatics
- ☆
Project supported by the National Natural Science Foundation of China (Nos. 61001129, 61179002), the State Key Laboratory Foundation (No. 9140c5302010802).