Abstract
We consider a problem of admission control to a single queue in discrete time. The controller has access to k step old queue lengths only, where k can be arbitrary. The problem is motivated, in particular, by recent advances in high-speed networking where information delays have become prominent. We formulate the problem in the framework of Completely Observable Controlled Markov Chains, in terms of a multi-dimensional state variable. Exploiting the structure of the problem, we show that under appropriate conditions, the multi-dimensional Dynamic Programming Equation (DPE) can be reduced to a unidimensional one. We then provide simple computable upper and lower bounds to the optimal value function corresponding to the reduced unidimensional DPE. These upper and lower bounds, along with a certain relationship among the parameters of the problem, enable us to deduce partially the structural features of the optimal policy. Our approach enables us to recover simply, in part, the recent results of Altman and Stidham, who have shown that a multiple-threshold-type policy is optimal for this problem. Further, under the same relationship among the parameters of the problem, we provide easily computable upper bounds to the multiple thresholds and show the existence of simple relationships among these upper bounds. These relationships allow us to gain very useful insights into the nature of the optimal policy. In particular, the insights obtained are of great importance for the problem of actually computing an optimal policy because they reduce the search space enormously.
Similar content being viewed by others
References
E. Altman and P. Nain, Closed-loop control with delayed information, Performance Evaluation Review 20 (1992) 193–204.
E. Altman and G. Koole, Stochastic scheduling games with Markov decision arrival processes, J. Comput. Math. Appl. (1993) 141–148.
E. Altman and G. Koole, Control of a random walk with noisy delayed information, Systems Control Lett. 24 (1995) 207–213.
E. Altman and S. Stidham, Optimality of monotonic policies for two-action Markovian decision processes, with applications to control of queues with delayed information, Report UNC/OR/TR–94–2, Univ. of North Carolina, Chapel Hill (1994).
D. Artiges, Optimal routing into two heterogeneous service stations with delayed information, IEEE Trans. Automat. Control 40 (1995) 1234–1236.
B. Hajek, Optimal control of two interacting service stations, IEEE Trans. Automat. Control 29 (1984) 491–499.
O. Hernández-Lerma, Adaptive Markov Control Processes (Springer, 1989).
G. Koole, Optimal repairman assignment in two symmetric maintenance models, European J. Oper. Res. 82 (1995) 295–301.
P.R. Kumar and P.P. Varaiya, Stochastic Systems: Estimation, Identification and Adaptive Control (Prentice-Hall, 1986).
J. Kuri and A. Kumar, Optimal control of arrivals to queues with delayed queue length information, in: Proc. of 31st IEEE Conference on Decision and Control (1992) pp. 991–992.
J. Kuri and A. Kumar, Optimal control of arrivals to queues with delayed queue length information, IEEE Trans. Automat. Control 40 (1995) 1444–1450.
J. Kuri, Optimal control problems in communication networks with information delays and quality of service constraints, Ph.D. thesis, Indian Institute of Science (1995).
D. Stoyan, Comparison Methods for Queues and Other Stochastic Models (Wiley, 1983).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kuri, J., Kumar, A. On the optimal control of arrivals to a single queue with arbitrary feedback delay. Queueing Systems 27, 1–16 (1997). https://doi.org/10.1023/A:1019145611110
Issue Date:
DOI: https://doi.org/10.1023/A:1019145611110