Skip to main content
SpringerLink
Log in

Stabilization of stochastic systems with hidden Markovian jumps

  • Scientific Papers
  • Published:
Science in China Series : Information Sciences Aims and scope Submit manuscript

Abstract

This paper considers the adaptive control of discrete-time hybrid stochastic systems with unknown randomly jumping parameters described by a finite-state hidden Markov chain. An intuitive yet longstanding conjecture in this area is that such hybrid systems can be adaptively stabilized whenever the rate of transition of the hidden Markov chain is small enough. This paper provides a rigorous positive answer to this conjecture by establishing the global stability of a gradient-algorithm-based adaptive linear-quadratic control.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Elliott, R. J., Aggoun, L., Moore, J. B., Hidden Markov Models, Estimation and Control, New York: Springer-Verlag, 1995.

    MATH  Google Scholar 

  2. Krasovskii, N. N., Lidskii, E. A., Analytic design of controller in systems with random attributes, Part I–III, Automat. Remote Contr., 1961, 22: 1021; 1141; 1289.

    MathSciNet  Google Scholar 

  3. Sworder, D. D., Feedback control of a class of linear systems with jump parameters, IEEE Trans. Automatic Contr., 1969, AC-14: 9.

    Article  MathSciNet  Google Scholar 

  4. Wonham, W. M., Random differential equations in control theory, Probabilistic Methods in Applied Mathematics, Vol II, (ed. Bharucha, A. T.), New York: Academic, 1971, 131–213.

    Google Scholar 

  5. Blair, W. P. Jr., Sworder, D. D., Feedback control of linear discrete systems with jump parameters and quadratic criteria, Int. J. Contr., 1975, 21(5): 833.

    Article  MATH  MathSciNet  Google Scholar 

  6. Chizeck, H. J., Willsky, A. S., Castanon, D., Discreet-time Markovian-jump linear quadratic optimal control, Int. J. Contr., 1986, 43(1): 231.

    Article  MathSciNet  Google Scholar 

  7. Ji, Y. D., Chizeck, H. J., Controllability, stabilizability, and continuous-time Markovian jump linear quadratic control, IEEE Trans. Automat. Control, 1990, AC-35(7): 777.

    Article  MathSciNet  Google Scholar 

  8. Ji, Y. D., Chizeck, H. J., Jump linear quadratic Gaussian control: steady-state solution and testable conditions, Control Theory and Advanced Technology, 1990, 6(3): 289.

    MathSciNet  Google Scholar 

  9. Ji, Y. D., Chizeck, H. J., Controllability, observability, and discrete-time Markovian jump linear quadratic control, Int. J. Contr., 1988, 48(2): 481.

    Article  MATH  MathSciNet  Google Scholar 

  10. Mariton, M., Bertrand, P., Output feedback for a class of linear systems with stochastic parameters, IEEE Trans. Automat. Control, 1985, AC-30(9): 898.

    Article  MathSciNet  Google Scholar 

  11. Caines, P. E., Chen, H. F., Optimal adaptive LQG control for systems with finite state process parameters, IEEE Trans. Automat. Control, 1985, AC-30: 185.

    Article  MathSciNet  Google Scholar 

  12. Caines, P. E., Zhang, J. F., On the adaptive control of jump parameter systems via nonlinear filtering, SIAM J. Control and Optimization, 1995, 33(6): 1758.

    Article  MATH  MathSciNet  Google Scholar 

  13. Dufour, F., Bertrand, P., Stabilizing control law for hybrid models, IEEE Transactions on Automatic Control, 1994, 39 (11): 2354.

    Article  MATH  MathSciNet  Google Scholar 

  14. Sworder, D. D., Hybrid adaptive control, Appl. Math. Comp., 1991, 45: 173.

    Article  MATH  MathSciNet  Google Scholar 

  15. Everdij, M. H. C., Blom, H. A. P., Embedding adaptive JLQG into LQ martingale control with a complete observable stochastic control matrix, IEEE Trans. Automat. Control, 1996, AC-41: 424.

    MathSciNet  Google Scholar 

  16. Dufour, F., Elliott, R. J., Adaptive control of linear systems with Markov perturbations, IEEE Trans. Automat. Control, 1998, AC-43: 351.

    Article  MathSciNet  Google Scholar 

  17. Guo, L., On adaptive stabilization of time-varying stochastic systems, SIAM J. Control and Optimization, 1990, 28(6): 1432.

    Article  MATH  MathSciNet  Google Scholar 

  18. Ford, J. J., Moore, J. B., Adaptive estimation of HMM transition probability, IEEE Trans. Signal Processing, 1998, SP-46: 1374.

    Article  Google Scholar 

  19. Guo, L., Ljung, L., Wang, G. J., Necessary and sufficient condition for stability of LMS, IEEE Trans. Automat. Control, 1997: 761.

  20. Ioannou, P. A., Sun, J., Robust Adaptive Control, Upper Saddle River, NJ: Prenticce-Hall, 1996.

    Google Scholar 

  21. Naik, S. M., Kumar, P. R., Robust indirect adaptive control of time-varying plants with unmodeled dynamics and disturbances, SIAM J. Control and Optimization, 1994, 32(6): 1696.

    Article  MATH  MathSciNet  Google Scholar 

  22. Delchamps, D., Analytic feedback control and the algebraic Riccati equation, IEEE Trans. Automat. Control, 1984, AC-29: 1031.

    Article  Google Scholar 

  23. Doob, J. L., Stochastic Processes, New York: Wiley, 1953.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Systems Science, Chinese Academy of Sciences, 100080, Beijing, China

    Huang Minyi & Guo Lei

Authors
  1. Huang Minyi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guo Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guo Lei.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Huang, M., Guo, L. Stabilization of stochastic systems with hidden Markovian jumps. Sci China Ser F 44, 104–118 (2001). https://doi.org/10.1007/BF02713969

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02713969

Keywords

Search

Navigation