Skip to main content
SpringerLink
Log in

Central Limit Theorems for Stochastic Optimization Algorithms Using Infinitesimal Perturbation Analysis

  • Published:
Discrete Event Dynamic Systems Aims and scope Submit manuscript

Abstract

Central limit theorems are obtained for the “perturbation analysis Robbins-Monro single run” (PARMSR) optimization algorithm, with updates either after every L customers or after every busy period, in the context of the optimization of a GI/GI/1 queue. The PARMSR algorithm is a stochastic approximation (SA) method for the optimization of infinite-horizon models. It is shown that the convergence rate and the asymptotic variance constant of the optimization algorithm, as a function of the total computing budget (i.e., total number of customers), are the same for both updating methods, and independent of L, provided that the step sizes of SA are chosen in the (asymptotically) optimal way for each method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Asmussen, S. 1987. Applied Probability and Queues. New York: John Wiley and Sons.

    Google Scholar 

  • Chen, H. F. 1993. Asymptotic efficient stochastic approximation. Stochastics and Stochastics Reports 45: 1–16.

    Google Scholar 

  • Chen, H. F. 1994. Stochastic approximation and its new applications. Proc. 1994 Hong Kong International Workshop on New Directions in Control and Manufacturing, pp. 2-12.

  • Chen, H. F., and Zhu, Y. M. 1996. Stochastic Approximation. Shanghai Scientific and Technological Publishers.

  • Chong, E. K. P., and Ramadge, P. J. 1992. Convergence of recursive optimization algorithms using infinitesimal perturbation analysis estimates. Discrete Event Dynamic Systems: Theory and Appl. 1: 339-372.

    Google Scholar 

  • Chong, E. K. P., and Ramadge, P. J. 1993. Optimization of queues using an infinitesimal perturbation analysisbased stochastic algorithm with general updates times. SIAM J. Contr. Optim. 31: 698-732.

    Google Scholar 

  • Chong, E. K. P., and Ramadge, P. J. 1994. Stochastic optimization of regenerative systems using infinitesimal perturbation analysis. IEEE Trans. Automat. Contr. AC-39: 1400-1410.

    Google Scholar 

  • Chow, Y. S. 1965. Local convergence of martingale and the law of large numbers. Ann. Math. Stat. 36: 552-558.

    Google Scholar 

  • Chow, Y. S., and Teicher, H. 1988. Probability Theory: Independence, Interchangeability, Martingale, second edition. New York: Springer-Verlag.

    Google Scholar 

  • Fu, M. C. 1990. Convergence of a stochastic approximation algorithm for the GI/G/1 queue using infinitesimal perturbation analysis. J. of Optim. Theory and Appl. 65: 149-160.

    Google Scholar 

  • Fu, M. C., and Ho, Y. C. 1988. Using perturbation analysis for gradient estimation, averaging, and updating in a stochastic approximation algorithm. Proc. of the Winter Simulation Conference, pp. 509-517.

  • Glasserman, P. 1991. Gradient Estimation via Perturbation Analysis. Boston: Kluwer.

    Google Scholar 

  • Glynn, P. W., and Whitt, W. 1992. The asymptotic efficiency of simulation estimators. Operations research 40: 505-520.

    Google Scholar 

  • Gut, A. 1988. Stopped Random Walks: Limit Theorems and Applications. New York: Springer-Verlag.

    Google Scholar 

  • Hall, P., and Heyde, C. C., 1980. Martingale Limit Theory and Its Application. New York: Acad. Press.

    Google Scholar 

  • Ho, Y. C., and Cao, X. R. 1983. Perturbation analysis and optimization of queueing networks. J. Optimiz. Theory Appl. 40: 559-582.

    Google Scholar 

  • Ho, Y. C., and Cao, X. R. 1991. Perturbation Analysis of Discrete Event Dynamic Systems. Boston: Kluwer.

    Google Scholar 

  • Kushner, H. J., and Clark, D.S. 1978. Stochastic Approximation Methods for Constrained and Unconstrained Systems. New York: Springer.

    Google Scholar 

  • Kushner, H. J., and Vázquez-Abad, F. J. 1996. Stochastic approximation methods for systems of interest over an infinite horizon. SIAM J. on Control and Optim. 34: 712-756.

    Google Scholar 

  • L'Ecuyer, P. 1990. A unified view of the IPA, SF, and LR gradient estimation techniques. Management Science 36: 1364-1383.

    Google Scholar 

  • L'Ecuyer, P., and Glynn, P. W. 1994. Stochastic optimization by simulation: convergence proofs for the GI/G/1 queue in steady-state. Management Sci. 40: 1562-1578.

    Google Scholar 

  • L'Ecuyer, P., Giroux, N., and Glynn, P. W. 1994. Stochastic optimization by simulation: numerical experiments with the M/M/1 queue in steady-state. Management Sci. 40: 1245-1261.

    Google Scholar 

  • L'Ecuyer, P., and Yin, G. 1998. Budget-dependent convergence rate of stochastic approximation. SIAM Journal on Optimization 8: 217-247.

    Google Scholar 

  • Meketon, M. S. 1987. Optimization in simulation: a survey of recent results. Proc. of the 1987 Winter Simulation Conference, pp. 58-67.

  • Nevel'on, M. B., and Hasminsky, R. S. 1972. Stochastic approximation and recursive estimation. Translated in Amer. Math. Soc. Transl. Monographs, vol. 24. Providence.

  • Nummelin, E. 1981. Regeneration in tandem queues. Adv. Appl. Prob. 13: 221-230.

    Google Scholar 

  • Polyak, B. T. 1990. New method of stochastic approximation type procedure. Automation and Remote Control 51: 937-946.

    Google Scholar 

  • Robbins, H., and Monro, S. 1951. A stochastic approximation method. Ann. Math. Statist. 22: 400-407.

    Google Scholar 

  • Sigman, K. 1990. Notes on the stability of closed queueing networks. J. Appl. Probab. 26: 678-682. 1990. Corrections. J. Appl. Probab. 27: 735.

    Google Scholar 

  • Sigman, K. 1990. The stability of open queueing networks. Stoch. Processes and their Appls. 35: 11-25.

    Google Scholar 

  • Stout, W. F. 1974. Almost Sure Convergence. New York: Academic Press.

    Google Scholar 

  • Suri, R. 1989. Perturbation analysis: the state of the art and research issues explained via the GI/G/1 queue. Proc. IEEE 77: 114-137.

    Google Scholar 

  • Suri, R., and Leung, Y. T. 1989. Single run optimization of discrete event simulations-An empirical study using the M/M/1 queue. IIE Transactions 21: 35-49.

    Google Scholar 

  • Suri, R., and Zazanis, M.A. 1988. Perturbation analysis gives strongly consistent sensitivity estimates for the M/G/1 queue. Management Sci. 34: 39-64.

    Google Scholar 

  • Tang, Q. Y., and Chen, H. F. 1994. Convergence of perturbation analysis based optimization algorithm with fixed number of customers period. Discrete Event Dynamic Systems: Theory and Appl. 4: 359-375.

    Google Scholar 

  • Tang, Q. Y., Chen, H. F., and Han, Z. J. 1997. Convergence rates of Perturbation-Analysis-Robbins-Monro-Single-Run algorithms for single server queues. IEEE Trans. on Autom. Contr. 42: 1442-1447.

    Google Scholar 

  • Tang, Q. Y., L'Ecuyer, P., and Chen, H. F. 1999. Asymptotic efficiency of perturbation analysis-based stochastic approximation with averaging. SIAM J. on Control and Optimization 37: 1822-1847.

    Google Scholar 

  • Thorisson, H. (1985). The queue GI=G=1: finite moments of the cycle variables and uniform rates of convergence. Stoch. Processes and their Appl. 19: 85-99.

    Google Scholar 

  • Wolff, R. 1989. Stochastic Modeling and the Theory of Queues. NJ: Prentice-Hall.

    Google Scholar 

  • Yin, G. 1991. On extension of Polyak' averaging approach to stochastic approximation. Stochastics and Stochastics Reports 36: 245-264.

    Google Scholar 

Download references

Authors
  1. Qian-Yu Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Han-Fu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tang, QY., Chen, HF. Central Limit Theorems for Stochastic Optimization Algorithms Using Infinitesimal Perturbation Analysis. Discrete Event Dynamic Systems 10, 5–32 (2000). https://doi.org/10.1023/A:1008347431468

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008347431468

Search

Navigation