Abstract
We review techniques for optimizing stochastic discrete-event systems via simulation. We discuss both the discrete parameter case and the continuous parameter case, but concentrate on the latter which has dominated most of the recent research in the area. For the discrete parameter case, we focus on the techniques for optimization from a finite set: multiple-comparison procedures and ranking-and-selection procedures. For the continuous parameter case, we focus on gradient-based methods, including perturbation analysis, the likelihood ratio method, and frequency domain experimentation. For illustrative purposes, we compare and contrast the implementation of the techniques for some simple discrete-event systems such as the (s, S) inventory system and theGI/G/1 queue. Finally, we speculate on future directions for the field, particularly in the context of the rapid advances being made in parallel computing.
Similar content being viewed by others
References
V.M. Aleksandrov, V.I. Sysoyev and V.V. Shemeneva, Stochastic optimization, Eng. Cybern. 5(1968)11–16.
A.E. Albert and L.A. Gardner, Jr.,Stochastic Approximation and Nonlinear Regression, Research Monograph No. 42 (M.I.T. Press, Cambridge, MA, 1967).
S. Andradóttir, A new algorithm for stochastic approximation,Proc. 1990 Winter Simulation Conf. (1990) pp. 364–366.
F. Azadivar and J.J. Talavage, Optimization of stochastic simulation models, Math. Comp. Simul. 22(1980)231–241.
J. Banks and J.S. Carson,Discrete-Event System Simulation (Prentice-Hall, Englewood Cliffs, N.J., 1984).
R.E. Bechhofer, A single-sample multiple decision procedure for ranking means of normal populations with known variances, Ann. Math. Statist. 25(1954)16–39.
A. Benveniste, M. Metivier and P. Priouret,Adaptive Algorithms and Stochastic Approximation (Springer, New York, 1990).
W.E. Biles and J.J. Swain,Optimization and Industrial Experimentation (Wiley-Interscience, New York, 1980).
R.E. Bixby, J.W. Gregory, I.J. Lustig, R.E. Marsten and D.F. Shanno, Very large-scale programming: a case study in combining interior point and simplex methods, Oper. Res. 25(1992)885–897.
G.E.P. Box and N.R. Draper,Empirical Model-Building and Response Surfaces (Wiley, New York, 1987).
P. Brémaud and F. Vázquez-Abad, On the pathwise computation of derivatives with respect to the rate of a point process, Queueing Syst. 10(1993)249–270.
M. Caramanis and G. Liberopoulos, Perturbation analysis for the design of flexible manufacturing flow controllers, Oper. Res. 40(1992)1107–1126.
G. Liberopolous and M. Caramanis, Infiniresimal perturbation analysis for second derivative estimation and design of manufacturing flow controllers, J. Optim. Theory Appl. (1994), to appear.
C.G. Cassandras and S.G. Strickland, On-line sensitivity analysis of Markov chains, IEEE Trans. Auto. Control AC-34(1989)76–86.
E.K.P. Chong and P.J. Ramadge, Convergence of recursive optimization algorithms using infinitesimal perturbation analysis, Discr. Event Dyn. Syst. 1(1992)339–372.
E.K.P. Chong and P.J. Ramadge, Optimization of queues using an IPA based stochastic algorithm with general update times, SIAM J. Control Optim. (1993), to appear.
K.L. Chung, On a stochastic approximation method, Ann. Math. Statist. 25(1954)463–483.
W.G. Cochran and G.M. Cox,Experimental Designs, 2nd ed. (Wiley, New York, 1957).
E.J. Dudewicz and S.R. Dalal, Allocation of measurements in ranking and selection with unequal variances, Sankhya B37(1975)28–78.
P. Dupuis and R. Simha, On sampling controlled stochastic approximation, IEEE Trans. Auto. Control AC-36(1991)915–924.
W. Farrell, Literature review and bibliography of simulation optimization,Proc. Winter Simulation Conf. (1977) pp. 117–124.
M.C. Fu, Convergence of a stochastic approximation algorithm for theGI/G/1 queue using infinitesimal perturbation analysis, J. Optim. Theory Appl. 65(1990)149–160.
M.C. Fu, Sample path derivatives for (s, S) inventory systems, Oper. Res. (1993), to appear.
M.C. Fu and K. Healy, Simulation optimization of (s, S) inventory systems,Proc. Winter Simulation Conf. (1992) pp. 506–514.
M.C. Fu and Y.C. Ho, Using perturbation analysis for gradient estimation, averaging, and updating in a stochastic approximation algorithm,Proc. Winter Simulation Conf. (1988) pp. 509–517.
M.C. Fu and J.Q. Hu, Extensions and generalizations of smoothed perturbation analysis in a generalized semi-Markov process framework, IEEE Trans. Auto. Control AC-37(1992)1483–1500.
M.C. Fu and J.Q. Hu, On choosing the characterization for smoothed perturbation analysis, IEEE Trans. Auto. Control AC-36(1991)1331–1336.
M.C. Fu and J.Q. Hu, Second derivative sample path estimators for theGI/G/m queue, Manag. Sci. 39(1993)359–383.
M.C. Fu, J.Q. Hu and L. Shi, Likelihood ratio methods with conditional Monte Carlo and splitting,Proc. 1993 Allerton Conf., to appear.
R. Fujimoto, Parallel discrete event simulation, Commun. ACM 33(1990)30–53.
A. Gaivoronski, Optimization of stochastic discrete event dynamic systems: a survey of some recent results, Lecture Notes in Economics and Mathematical Systems 374,Proc. Workshop on Simulation and Optimization, Laxenburg, Austria, ed. G. Pflug and U. Dieter (1992) pp. 24–44.
P. Glasserman,Gradient Estimation Via Perturbation Analysis (Kluwer Academic, 1991).
P. Glasserman, Structural conditions for perturbation analysis derivative estimation: finite-time performance indices, Oper. Res. 39(1991)724–738.
P. Glasserman and W.B. Gong, Smoothed perturbation analysis for a class of discrete event systems, IEEE Trans. Auto. Control AC-35(1990)1218–1230.
P. Glasserman and D.D. Yao, Some guidelines and guarantees for common random numbers, Manag. Sci. 38(1992)884–908.
P.W. Glynn, Likelihood ratio gradient estimation: an overview,Proc. Winter Simulation Conf. (1987) pp. 366–374.
P.W. Glynn, Likelihood ratio gradient estimation for stochastic systems, Commun. ACM 33(1990)75–84.
P.W. Glynn, Optimization of stochastic systems,Proc. Winter Simulation Conf. (1986) pp. 52–59.
P.W. Glynn, Optimization of stochastic systems via simulation,Proc. Winter Simulation Conf. (1989) pp. 90–105.
P.W. Glynn, Stochastic approximation for Monte Carlo optimization,Proc. Winter Simulation Conf. (1986) pp. 356–364.
P.W. Glynn and P. Heidelberger, Analysis of parallel replicated simulations under a completion time constraint, ACM Trans. Modeling Comp. Simul. 1(1991)3–23.
D. Goldsman, B.L. Nelson and B. Schmeiser, Methods for selecting the best system,Proc. Winter Simulation Conf. (1991) pp. 177–186.
W.B. Gong and Y.C. Ho, Smoothed perturbation analysis of discrete-event dynamic systems, IEEE Trans. Auto. Control AC-32(1987)858–867.
S.S. Gupta, On a decision rule for ranking means, Inst. Statist. Mimeo. Ser. No. 150, University of North Carolina, Chapel Hill, NC (1956).
J. Haddock and G. Bengu, Application of a simulation optimization system for a continuous review inventory model,Proc. Winter Simulation Conf. (1987) pp. 383–390.
K. Healy and L.W. Schruben, Retrospective simulation response optimization,Proc. Winter Simulation Conf. (1991) pp. 901–906.
K. Healy, Retrospective simulation response optimization, Ph.D. Dissertation, Cornell University (1992).
Y.C. Ho, Special issue on dynamics of discrete event systems, Proc. IEEE 77 (1989).
Y.C. Ho, Perturbation analysis: concepts and algorithms,Proc. Winter Simulation Conf. (1992) pp. 231–240.
Y.C. Ho amd X.R. Cao,Discrete Event Dynamic Systems and Perturbation Analysis (Kluwer Academic, 1991).
Y.C. Ho and X.R. Cao, Optimization and perturbation analysis of queueing networks, J. Optim. Theory Appl. 40(1983)559–582.
Y.C. Ho, X.R. Cao and C.G. Cassandras, Infinitesimal and finite perturbation analysis for queueing networks, Automatica 19(1983)439–445.
Y.C. Ho and C. Cassandras, A new approach to the analysis of discrete event dynamic systems, Automatica 19(1983)149–167.
Y.C. Ho, A. Eyler and T.T. Chien, A gradient technique for general buffer storage design in a serial production line, Int. J. Prod. Res. 17(1979)557–580.
Y.C. Ho, M.A. Eyler and T.T. Chien, A new approach to determine parameter sensitivity of transfer lines, Manag. Sci. 29(1983)700–714.
Y.C. Ho and S. Li, Extensions of perturbation analysis of discrete-event dynamic systems, IEEE Trans. Auto. Control AC-33(1988)427–438.
Y.C. Ho, S. Li and P. Vakili, On the efficient generation of discrete event sample paths under different system parameters, Math. Comp. Simul. 30(1988)347–370.
Y.C. Ho, L. Shi, L. Dai and W.B. Gong, Optimizing discrete event systems via the gradient surface method, Discr. Event Dyn. Syst. 2(1992)99–120.
Y.C. Ho, R. Sreenevas and P. Vakili, Ordinal optimization of DEDS, Discr. Event Dyn. Syst. 2(1992)61–88.
Y. Hochberg and A.C. Tamhane,Multiple Comparison Procedures (Wiley, New York, 1987).
J.C. Hsu, Simultaneous confidence levels for all distributions from the best, Ann. Statist. 9(1981)1026–1034.
J.C. Hsu and B.L. Nelson, Optimization over a finite number of system designs with one-stage sampling and multiple comparisons with the best,Proc. Winter Simulation Conf. (1988) pp. 451–457.
J.S. Hunter and T.H. Naylor, Experimental designs for computer simulation experiments, Manag. Sci. 16(1970)422–434.
S.H. Jacobson, Optimal mean squared error analysis of the harmonic gradient estimators, J. Optim. Theory Appl. (1994), to appear.
S.H. Jacobson, Oscillation amplitude consideration in frequency domain experiments,Proc. Winter Simulation Conf. (1989) pp. 406–410.
S.H. Jacobson, Variance and bias reduction techniques for harmonic gradient estimators, Appl. Math. Comp. (1993), to appear.
S.H. Jacobson and L.W. Schruben, A review of techniques for simulation optimization, Oper. Res. Lett. 8(1989)1–9.
S.H. Jacobson and L.W. Schruben, A simulation optimization procedure using harmonic analysis, Oper. Res. (1992), submitted.
S.H. Jacobson, A. Buss and L.W. Schruben, Driving frequency selection for frequency domain simulation experiments, Oper. Res. 39(1991)917–924.
S.H. Jacobson, D. Morrice and L.W. Schruben, The global simulation clock as the frequency domain experiment index,Proc. Winter Simulation Conf. (1988) pp. 558–563.
H. Kesten, Accelerated stochastic approximation, Ann. Math. Statist. 29(1958)41–59.
A.I. Khuri and J.A. Cornell,Response Surfaces (Marcel Dekker, 1987).
J. Kiefer and J. Wolfowitz, Stochastic estimation of the maximum of a regression function, Ann. Math. Statist. 23(1952)462–466.
J.P.C. Kleijnen,Statistical Tools for Simulation Practitioners (Marcel Dekker, New York, 1987).
H.J. Kushner and D.C. Clark,Stochastic Approximation Methods for Constrained and Unconstrained Systems (Springer, New York, 1978).
A.M. Law and W.D. Kelton,Simulation Modeling and Analysis, 2nd ed. (McGraw-Hill, New York, 1991).
P. L'Ecuyer, A unified view of the IPA, SA, and LR gradient estimation techniques, Manag. Sci. 36(1990)1364–1383.
P. L'Ecuyer, An overview of derivative estimation,Proc. Winter Simulation Conf. (1991) pp. 207–217.
P. L'Ecuyer, Convergence rates for steady-state derivative estimators, Ann. Oper. Res. 39(1992)121–136.
P. L'Ecuyer and P.W. Glynn, Stochastic optimization by simulation: convergence proofs for theGI/G/1 queue in steady state, Manag. Sci. (1994), to appear.
P. L'Ecuyer, N. Giroux and P.W. Glynn, Stochastic optimization by simulation: numerical experiments with a simple queue in steady state, Manag. Sci. (1994), to appear.
P. L'Ecuyer and G. Perron, On the convergence rates of IPA and FDC derivative estimators for finite-horizon stochastic simulations, Oper. Res. (1993), submitted.
Y.T. Leung and R. Suri, Finite-time behavior of two simulation optimization algorithms,Proc. Winter Simulation Conf. (1990) pp. 372–376.
L. Ljung, Analysis of recursive stochastic algorithms, IEEE Trans. Auto. Control AC-22(1977)551–575.
L. Ljung, Strong convergence of a stochastic approximation algorithm, Ann. Statist. 6(1978)680–696.
M. Mitra and S.K. Park, Solution to the indexing problem of frequency domain simulation experiments,Proc. Winter Simulation Conf. (1991) pp. 907–915.
R.H. Myers, A.I. Khuri and W.H. Carter, Response surface methodology: 1966–1988, Technometrics 31(1989)137–157.
R.H. Myers,Response Surface Methodology (Allyn and Bacon, Boston, 1971).
M.B. Nevelson and R. Zalmanovich, Stochastic approximation and recursive estimation, translation of mathematical monographs, Vol. 47 (1973).
G.Ch. Pflug, On-line optimization of simulated Markovian processes, Math. Oper. Res. 15(1990)381–395.
G.Ch. Pflug, Sampling derivatives of probabilities, Computing 42(1989)315–328.
B.T. Polyak, New method of stochastic approximation type, Auto. Remote Control 51(1990)937–946.
M.I. Reiman and A. Weiss, Sensitivity analysis via likelihood ratios,Proc. Winter Simulation Conf. (1986) pp. 285–289.
M.I. Reiman and A. Weiss, Sensitivity analysis for simulations via likelihood ratios, Oper. Res. 37(1989)830–844.
H. Robbins and S. Monro, A stochastic approximation method, Ann. Math. Statist. 22(1951)400–407.
K.W. Ross and J. Wang, Solving product form stochastic networks with Monte Carlo summation,Proc. Winter Simulation Conf. (1990) pp. 270–275.
R.Y. Rubinstein, How to optimize discrete-event systems from a single sample path by the score function method, Ann. Oper. Res. 27(1991)175–212.
R.Y. Rubinstein, The push-out method for sensitivity analysis of discrete event systems, Ann. Oper. Res. (1993), to appear.
R.Y. Rubinstein, Sensitivity analysis and performance extrapolation for computer simulation models, Oper. Res. 37(1989)72–81.
R.Y. Rubinstein,Monte Carlo Optimization: Simulation and Sensitivity of Queueing Networks (Wiley, New York, 1986).
R.Y. Rubinstein and A. Shapiro,Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization by the Score Function Method (Wiley, 1992).
D. Ruppert, Almost sure approximations to the Robbins-Monro and Kiefer-Wolfowitz processes with dependent noise, Ann. Statist. 16(1982)178–187.
A. Ruszczynski and W. Syski, Stochastic approximation method with gradient averaging for unconstrained problems, IEEE Trans. Auto. Control AC-28(1983)1097–1105.
J. Sacks, Asymptotic distribution of stochastic approximation procedures, Ann. Math. Statist. 29(1958)373–405.
M.H. Safizadeh, Optimization in simulation: current issues and the future outlook, Naval Res. Logist. Quart. 37(1990)807–825.
M.H. Safizadeh and B.M. Thornton, Optimization in simulation experiments using response surface methodology, Comp. Ind. Eng. 8(1984)11–27.
J. Santner and A.C. Tamhane,Design of Experiments: Ranking and Selection (Marcel Dekker, 1984).
R.G. Sargent, Research issues in metamodeling,Proc. Winter Simulation Conf. (1991) pp. 888–893.
R.G. Sargent and T.K. Som, Current issues in frequency domain experimentation, Manag. Sci. 38(1992)667–687.
L.W. Schruben, Simulation optimization using frequency domain methods,Proc. Winter Simulation Conf. (1986) pp. 366–369.
L.W. Schruben and V.J. Cogliano, An experimental procedure for simulation response surface model identification, Commun. ACM 30(1987)716–730.
L.W. Schruben and V.J. Cogliano, Simulation sensitivity analysis: a frequency domain approach,Proc. Winter Simulation Conf. (1981) pp. 455–459.
B. Schmeiser, Simulation output data analysis, Ann. Oper. Res. (1994), this volume.
L.Y. Shi, Discontinuous perturbation analysis of discrete event dynamic systems, IEEE Trans. Auto. Control (1993), submitted.
D.E. Smith, An empirical investigation of optimum-seeking in the computer simulation situation, Oper. Res. 21(1973)475–497.
D.E. Smith, Automatic optimum-seeking program for digital simulation, Simulation 27(1976)27–32.
D.E. Smith, Requirements of an optimizer for computer simulation, Naval Res Logist. Quart. 20(1973)161–179.
D.W. Sullivan and J.R. Wilson, Restricted subset selection procedures for simulation, Oper. Res. 37(1989)52–71.
R. Suri, Perturbation analysis: the state of the art and research issues explained via theGI/G/1 queue, Proc. IEEE 77(1989)114–137.
R. Suri and Y.T. Leung, Single run optimization of discrete event simulations — an empirical study using theM/M/1 queue, IIE Trans. 21(1991)35–49.
R. Suri and M. Zazanis, Perturbation analysis gives strongly consistent sensitivity estimates for theM/G/1 queue, Manag. Sci. 34(1988)39–64.
P. Vakili, Using a standard clock technique for efficient simulation, Oper. Res. Lett. (1991) 445–452.
P. Vakili, Massively parallel and distributed simulation of a class of discrete event systems: a different perspective, ACM Trans. Modeling Comp. Simul. (1993), to appear.
F. Vázquez-Abad and P. L'Ecuyer, Comparing alternative methods for derivative estimation when IPA does not apply directly,Proc. Winter Simulation Conf. (1991) pp. 1004–1011.
Y. Wardi, Simulation-based stochastic algorithm for optimizingGI/G/1 queues, Manuscript, Department of Industrial Engineering, Ben Gurion University (1988).
M.T. Wasan,Stochastic Approximation (Cambridge University Press, 1969).
J.R. Wilson, Future directions in response surface methodology for simulation,Proc. Winter Simulation Conf. (1987) pp. 378–381.
S. Yakowitz, A globally convergent stochastic approximation, SIAM J. Control Optim. (1993), to appear.
S. Yakowitz, T. Jayawadena and S. Li, Theory for automatic learning under partially observed Markov-dependent noise, IEEE Trans. Auto. Control AC-37(1992)1316–1324.
W.N. Yang and B.L. Nelson, Using common random numbers and control variates in multiple-comparison procedures, Oper. Res. 39(1991)583–591.
G. Yin, Stochastic approximation via averaging: the Polyak approach revisited, Lecture Notes in Economics and Mathematical Systems Vol. 374,Proc. Workshop on Simulation and Optimization, Laxenburg, Austria, ed. G. Pflug and U. Dieter (1990) pp. 119–134.
M. Zazanis and R. Suri, Estimating first and second derivatives of response time forG/G/1 queues from a single sample path, Technical Report, Northwestern University, Evanston, IL; also Queueing Syst. (1986).
B. Zhang and Y.C. Ho, Performance gradient estimation for very large finite Markov chains, IEEE Trans. Auto. Control AC-36(1991)1218–1227.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fu, M.C. Optimization via simulation: A review. Ann Oper Res 53, 199–247 (1994). https://doi.org/10.1007/BF02136830
Issue Date:
DOI: https://doi.org/10.1007/BF02136830