Abstract
The problem of admission control of packets in communication networks is studied in the continuous time queueing framework under different classes of service and delayed information feedback. We develop and use a variant of a simulation based two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm for finding an optimal feedback policy within the class of threshold type policies. Even though SPSA has originally been designed for continuous parameter optimization, its variant for the discrete parameter case is seen to work well. We give a proof of the hypothesis needed to show convergence of the algorithm on our setting along with a sketch of the convergence analysis. Extensive numerical experiments with the algorithm are illustrated for different parameter specifications. In particular, we study the effect of feedback delays on the system performance.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D.P. Bertsekas, Dynamic Programming and Optimal Control, second edition (Athena Scientific, Belmont, MA, 2001).
D.P. Bertsekas and R. Gallager Data Networks (Prentice Hall, Englewood Cliffs, NJ, 1992).
D.P. Bertsekas and J.N. Tsitsiklis, Neuro-Dynamic Programming (Athena Scientific, Belmont, MA, 1996).
S. Bhatnagar and V.S. Borkar, Multiscale stochastic approximation for parametric optimization of hidden Markov models, Probability in the Engineering and Informational Sciences 11 (1997) 509–522.
S. Bhatnagar and V.S. Borkar, A two time scale stochastic approximation scheme for simulation based parametric optimization, Probability in the Engineering and Informational Sciences 12 (1998) 519–531.
S. Bhatnagar and V.S. Borkar, Multiscale chaotic SPSA and smoothed functional algorithms for simulation optimization, Simulation: Transactions of the Society for Modeling and Simulation International 79(10) (2003) 568–580.
S. Bhatnagar, M.C. Fu, S.I. Marcus and S. Bhatnagar, Two timescale algorithms for simulation optimization of hidden Markov models, HE Transactions 33(3) (2001) 245–258.
S. Bhatnagar, M.C. Fu, S.I. Marcus and P.J. Fard, “Optimal structured feedback policies for ABR flow control using two-timescale SPSA, IEEE/ACM Transactions on Networking 9(4) (2001) 479–491.
S. Bhatnagar, M.C. Fu, S.I. Marcus and I.-J. Wang, Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences, ACM Transactions on Modelling and Computer Simulation 13(2) (2003) 180–209.
O. Brandiere, Some pathological traps for stochastic approximation, SIAM J. Contr. and Optim. 36 (1998) 1293–1314.
H.F. Chen and T.E. Duncan and B. P.-Duncan, A Kiefer-Wolfowitz algorithm with randomized differences, IEEE Trans. Autom. Cont. 44(3) (1999) 442–453.
R.-G. Cheng, C.-J. Chang and L.-F. Lin, A QoS provisioning neural fuzzy connection admission controller for multimedia high speed networks, IEEE/ACM Trans, on Network. 7(1) (1999) 111–121.
E.K.P. Chong and P.J. Ramadge, Optimization of queues using an infinitesimal perturbation analysis-based stochastic algorithm with general update times, SIAM J. Contr. and Optim. 31(3) (1993) 698–732.
E.K.P. Chong and P.J. Ramadge, Stochastic optimization of regenerative systems using infinitesimal perturbation analysis, IEEE Trans, on Autom. Contr. 39(7) (1994) 1400–1410.
M.C. Fu, Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis, J. Optim. Theo. Appl. 65 (1990) 149–160.
L. Gerencsér, S.D. Hill and Z. Vágó, Optimization over discrete sets via SPSA, in Proceedings of the IEEE Conference on Decision and Control (1999) pp. 1791–1795.
M. Grossglauser, S. Keshav and D.N.C. Tse, RCBR: A simple and efficient service for multiple time-scale traffic, IEEE Trans, on Network. 5(6) (1997) 741–755.
M.W. Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks 2 (1989) 331–349.
Y.-C. Ho and X.-R. Cao Perturbation Analysis of Discrete Event Dynamical Systems (Kluwer, Boston, 1991).
F.P. Kelly, P.B. Key and S. Zachary, Distributed admission control, IEEE Journal on Selected Areas in Communications 18 (2000) 2617–2628.
S. Keshav An Engineering Approach to Computer Networking (Addison-Wesley, New York, 1997).
T.-H. Lee, K.-C. Lai and S.-T. Duann, Design of a real-time admission controller for ATM Networks, IEEE/ACM Trans, on Network 4(5) (1996) 758–765.
J. Liebeherr, D.E. Wrege and D. Ferrari Exact admission control for networks with a bounded delay service, IEEE/ACM Trans, on Network 4(6) (1996) 885–901.
P. Marbach, O. Mihatsch and J.N. Tsitsiklis, Call admission control and routing in integrated service networks using neuro-dynamic programming, IEEE Journal on Selected Areas in Communications 18(2) (2000) 197–208.
R. Pemantle, Nonconvergence to unstable points in urn models and stochastic approximations, Annals of Prob. 18 (1990) 698–712.
M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming (John Wiley, New York, 1994).
P.J. Schweitzer, Perturbation theory and finite Markov chains, J. Appl. Prob. 5 (1968) 401–413.
J.C. Spall, Multivariate stochastic approximation using a simultaneous perturbation gradient approximation, IEEE Trans. Autom. Contr. 37(3) (1992) 332–341.
F.J. Vazquez-Abad and H.J. Kushner, Estimation of the derivative of a stationary measure with respect to a control parameter, J. Appl. Prob. 29 (1992) 343–352.
J. Walrand and P. Varaiya, High-Performance Computer Networks (Morgan Kauffman, San Mateo, CA, 2000).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bhatnagar, S., Reddy, I.B.B. Optimal Threshold Policies for Admission Control in Communication Networks via Discrete Parameter Stochastic Approximation. Telecommun Syst 29, 9–31 (2005). https://doi.org/10.1007/s11235-005-6629-5
Received:
Revised:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11235-005-6629-5