Abstract
This paper studies a stochastic linear quadratic (LQ) control problem in the infinite time horizon with Markovian jumps in parameter values. In contrast to the deterministic case, the cost weighting matrices of the state and control are allowed to be indinifite here. When the generator matrix of the jump process – which is assumed to be a Markov chain – is known and time-invariant, the well-posedness of the indefinite stochastic LQ problem is shown to be equivalent to the solvability of a system of coupled generalized algebraic Riccati equations (CGAREs) that involves equality and inequality constraints. To analyze the CGAREs, linear matrix inequalities (LMIs) are utilized, and the equivalence between the feasibility of the LMIs and the solvability of the CGAREs is established. Finally, an LMI-based algorithm is devised to slove the CGAREs via a semidefinite programming, and numerical results are presented to illustrate the proposed algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ait Rami, M. and El Ghaoui, L. (1996), LMI optimization for nonstandard Riccati equations arising in stochastic control, IEEE Transactions on Automatic Control, 41, 1666–1671.
Ait Rami, M., Moore, J. and Zhou, X.Y. (2001), Indefinite stochastic linear quadratic control and generalized differential Riccati equation, SIAM Journal on Control and Optimization, 40, 1296–1311.
Ait Rami, M. and Zhou, X.Y. (2000), Linear matrix inequalities, Riccati equations, indefinite stochastic quadratic controls, IEEE Transactions on Automatic Control, 45, 1131-1143.
Albert, A. (1969), Conditions for positive and nonnegative definiteness in terms of pseudoinverses, SIAM Journal on Applied Mathematics, 17, 434-440.
Björk, T. (1980), Finite dimensional optimal filters for a class of Itô-processes with jumping parameters, Stochastics, 4, 167–183.
Bensoussan, A. (1982), Lectures on stochastic control, Lecture Notes in Mathematics, 972, 1-62.
Boyd, S., El Ghaoui, L., Feron, E. and Balakrishnan, V., Linear Matrix Inequality in Systems and Control Theory, SIAM, Philadelphia, 1994.
Chen, S., Li X. and Zhou, X.Y. (1998), Stochastic linear quadratic regulators with indefinite control weight costs, SIAM Journal on Control and Optimization, 36, 1685-1702.
Davis, M.H.A. (1977), Linear Estimation and Stochastic Control, Chapman and Hall, London.
El Ghaoui, L. and Ait Rami, M. (1996), Robust state-feedback stabilization of jump linear systems via LMIs, International Journal of Robust and Nonlinear Control, 6, 1015-1022.
El Ghaoui, L., Nikoukhah, R. and Delebecque, F. (1995), LMITOOL: A front-end for LMI optimization in matlab. Available via anonymous ftp to ftp.ensta.fr, under /pub/elghaoui/lmitool.
Ji, Y. and Chizeck, H.J. (1990), Controllability, stabilizability, and continuous-time Markovian jump linear quadratic, IEEE Transactions on Automatic Control, 35, 777-788.
Ji, Y. and Chizeck, H.J. (1992), Jump linear quadratic Gaussian control in continuous-time, IEEE Transactions on Automatic Control, 37, 1884-1892.
Krasosvkii, N.N. and Lidskii, E.A. (1961), Analytical design of controllers in systems with random attributes I, II, III, Automation and Remote Control, 22, 1021-1025, 1141-1146, 1289-1294.
Mariton, M. (1990), Jump Linear Systems in Automatic Control, Marcel Dekker, New York.
Mariton, M. and Bertrand, P. (1985), A homotopy algorithm for solving coupled Riccati equations, Optimal Control Applications & Methods, 6, 351-357.
Nesterov, Y. and Nemirovsky, A. (1993), Interior point polynomial methods in convex programming: Theory and applications, SIAM.
Nikoukhah, R., Delebecque, F. and El Ghaoui, L. (1995), LMITOOL: A package for LMI optimization in scilab. INRIA Rocquencourt, France.
Penrose, R. (1955), A generalized inverse of matrices, Proceedings of the Cambridge Philosophical Society, 51, 406-413.
Porkolab, L. and Khachiyan, L. (1997), On the complexity of semidefinite programs, Journal of Global Optimization, 10, 351-365.
Vandenberghe, L. and Boyd, S. (1995), A primal-dual potential reduction method for problems involving matrix inequalities, Mathematical Programming, 69, 205-236.
Vandenberghe, L. and Boyd, S. (1996), Semidefinite programming, SIAM Review, 38, 49-95.
Wonham, W.M. (1968), On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6, 681-697.
Wonham,W.M. (1970), Random differential equations in control theory, Probabilistic Methods in Applied Mathematics, Academic, New York, 2, 131-212.
Zhang, Q. and Yin, G. (1999), On nearly optimal controls of hybrid LQG problems, IEEE Transactions on Automatic Control, 44, 2271–2282.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Li, X., Zhou, X.Y. & Ait Rami, M. Indefinite Stochastic Linear Quadratic Control with Markovian Jumps in Infinite Time Horizon. Journal of Global Optimization 27, 149–175 (2003). https://doi.org/10.1023/A:1024887007165
Issue Date:
DOI: https://doi.org/10.1023/A:1024887007165