Abstract
We introduce the theories and methodologies that utilize the special features of discrete event dynamic systems (DEDSs) for perturbation analysis (PA) and optimization of steady-state performance. Such theories and methodologies usually take different perspectives from the traditional optimization approaches and therefore may lead to new insights and efficient algorithms. The topic discussed includes the gradient-based optimization for systems with continuous parameters and the direct-comparison-based optimization for systems with discrete policies, which is an alternative to dynamic programming and may apply when the latter fails. Furthermore, these new insights can also be applied to continuous-time and continuous-state dynamic systems, leading to a new paradigm of optimal control.
This is a preview of subscription content, log in via an institution.
Bibliography
Baxter J, Bartlett PL (2001) Infinite-horizon policy-gradient estimation. J Artif Intell Res 15: 319–350
Cao XR (1985) Convergence of parameter sensitivity estimates in a stochastic experiment. IEEE Trans Autom Control 30:834–843
Cao XR (2005) A basic formula for online policy gradient algorithms. IEEE Trans Autom Control 50(5):696–699
Cao XR (2007) Stochastic learning and optimization – a sensitivity-based approach. Springer, New York
Cao XR, Wan YW (1998) Algorithms for sensitivity analysis of Markov systems through potentials and perturbation realization. IEEE Trans Control Syst Technol 6:482–494
Cao XR, Wan XW (2013) Analysis of non-linear behavior – a sensitivity-based approach. submitted
Cao XR, Wang DX, Lu T, Xu YF (2011) Stochastic control via direct comparison. Discret Event Dyn Syst Theory Appl 21:11–38
Cassandras CG, Lafortune S (1999) Introduction to discrete event systems. Kluwer Academic Publishers, Boston
Fang HT, Cao XR (2004) Potential-based on-line policy iteration algorithms for Markov decision processes. IEEE Trans Autom Control 49:493–505
Fu MC, Hu JQ (1997) Conditional Monte Carlo: gradient estimation and optimization applications. Kluwer Academic Publishers, Boston
Glasserman P (1991) Gradient estimation via perturbation analysis. Kluwer Academic Publishers, Boston
Heidelberger P, Cao XR, Zazanis M, Suri R (1988) Convergence properties of infinitesimal perturbation analysis estimates. Manag Sci 34:1281-1302
Ho YC, Cao XR (1983) Perturbation analysis and optimization of queueing networks. J Optim Theory Appl 40:559–582
Ho YC, Cao XR (1991) Perturbation analysis of discrete-event dynamic systems. Kluwer Academic Publisher, Boston
Marbach P, Tsitsiklis TN (2001) Simulation-based optimization of Markov reward processes. IEEE Trans Autom Control 46:191–209
Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York
Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22:400–407
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this entry
Cite this entry
Cao, CR. (2013). Perturbation Analysis of Steady-State Performance and Sensitivity-Based Optimization. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_57-1
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5102-9_57-1
Received:
Accepted:
Published:
Publisher Name: Springer, London
Online ISBN: 978-1-4471-5102-9
eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering
Publish with us
Chapter history
-
Latest
Perturbation Analysis of Steady-State Performance and Relative Optimization- Published:
- 04 January 2020
DOI: https://doi.org/10.1007/978-1-4471-5102-9_57-2
-
Original
Perturbation Analysis of Steady-State Performance and Sensitivity-Based Optimization- Published:
- 11 March 2014
DOI: https://doi.org/10.1007/978-1-4471-5102-9_57-1