Skip to main content
SpringerLink
Log in

Refined Estimation of the Bellman Function for Stochastic Optimal Control Problems with Probabilistic Performance Criterion

  • Stochastic Systems
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

In this paper, the optimal control problem for a discrete-time stochastic system with a general-form probabilistic criterion is considered. Using dynamic programming and the properties of the Bellman function, new two-sided bounds of this function that refine the earlier results are constructed. The derived bounds are then adopted to justify the application of the modified strategy that is optimal in the two-step investment portfolio management problem under risk to the corresponding multistep problem. An example that illustrates the advantages of such a strategy over other well-known strategies is given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Malyshev, V.V. and Kibzun, A.I., Analiz i sintez vysokotochnogo upravleniya letatel’nymi apparatami (Analysis and Design of High–Precision Aircraft Control), Moscow: Mashinostroenie, 1987.

    Google Scholar 

  2. Krasil’shchikov, M.N, Malyshev, V.V., and Fedorov, A.V., Autonomous Implementation of Dynamic Operations in a Geostationary Orbit. I. Formalization of Control Problem, J. Comp. Syst. Sci. Int., 2015, vol. 54, no. 6, pp. 916–930.

    Article  MathSciNet  MATH  Google Scholar 

  3. Malyshev, V.V., Starkov, A.V., and Fedorov, A.V., Optimal Control Design for Aircraft Keeping in an Orbit Group, Kosmonavt. Raketostr., 2012, no. 4, pp. 150–158.

    Google Scholar 

  4. Malyshev, V.V., Starkov, A.V., and Fedorov, A.V., The Combined Problem of Keeping and Evasion in a Neighborhood of a Reference Geostationary Orbit, Vestn. Mosk. Gorodsk. Pedagog. Univ., Ser. Ekon., 2013, no. 1, pp. 68–74.

    Google Scholar 

  5. Azanov, V.M. and Kan, Yu.S., Design of Optimal Strategies in the Problems of Discrete System Control by the Probabilistic Criterion, Autom. Remote Control, 2017, vol. 78, no. 6, pp. 1006–1027.

    Article  MathSciNet  MATH  Google Scholar 

  6. Azanov, V.M. and Kan, Yu.S., Bilateral Estimation of the Bellman Function in the Problems of Optimal Stochastic Control of Discrete Systems by the Probabilistic Performance Criterion, Autom. Remote Control, 2018, vol. 79, no. 2, pp. 203–215.

    Article  MathSciNet  MATH  Google Scholar 

  7. Kibzun, A.I. and Ignatov, A.N., On the Existence of Optimal Strategies in the Control Problem for a Stochastic Discrete Time System with Respect to the Probability Criterion, Autom. Remote Control, 2017, vol. 78, no. 10, pp. 1845–1856.

    Article  MathSciNet  MATH  Google Scholar 

  8. Kibzun, A.I. and Ignatov, A.N., Reduction of the Two–Step Problem of Stochastic Optimal Control with Bilinear Model to the Problem of Mixed Integer Linear Programming, Autom. Remote Control, 2016, vol. 77, no. 12, pp. 2175–2192.

    Article  MathSciNet  MATH  Google Scholar 

  9. Kibzun, A.I. and Ignatov, A.N., The Two–Step Problem of Investment Portfolio Selection from Two Risk Assets via the Probability Criterion, Autom. Remote Control, 2015, vol. 76, no. 7, pp. 1201–1220.

    Article  MathSciNet  MATH  Google Scholar 

  10. Kibzun, A.I. and Kuznetsov, E.A., Optimal Control of the Portfolio, Autom. Remote Control, 2001, vol. 62, no. 9, pp. 1489–1501.

    Article  MathSciNet  MATH  Google Scholar 

  11. Bunto, T.V. and Kan, Yu.S., Quantile Criterion–based Control of the Securities Portfolio with a Nonzero Ruin Probability, Autom. Remote Control, 2013, vol. 74, no. 5, pp. 811–828.

    Article  MathSciNet  MATH  Google Scholar 

  12. Grigor’ev, P.V. and Kan, Yu.S., Optimal Control of the Investment Portfolio with Respect to the Quantile Criterion, Autom. Remote Control, 2004, vol. 65, no. 2, pp. 319–336.

    Article  MathSciNet  MATH  Google Scholar 

  13. Vishnyakov, B.V. and Kibzun, A.I., A Two–Step Capital Variation Model: Optimization by Different Statistical Criteria, Autom. Remote Control, 2005, vol. 66, no. 7, pp. 1137–1152.

    Article  MathSciNet  MATH  Google Scholar 

  14. Jasour, A.M., Aybat, N.S., and Lagoa, C.M., Semidefinite Programming for Chance Constrained Optimization over Semialgebraic Sets, SIAM J. Optimization, 2015, no. 25 (3), pp. 1411–1440.

    Google Scholar 

  15. Jasour, A.M. and Lagoa, C.M., Convex Chance Constrained Model Predictive Control, arXiv preprint, arXiv:1603.07413, 2016.

    Book  Google Scholar 

  16. Jasour, A.M. and Lagoa, C.M., Convex Relaxations of a Probabilistically Robust Control Design Problem, Proc. 52nd IEEE Conf. on Decision and Control, 2013, pp. 1892–1897.

    Google Scholar 

  17. Hsieh, Ch.–H. and Barmish, B.R., On Drawdown–Modulated Feedback Control in Stock Trading, IFAC Proc., 2017, vol. 50, no. 1, pp. 952–958.

    Google Scholar 

  18. Hsieh, Ch.–H., Barmish B.R., and Gubner, J., At What Frequency Should the Kelly Bettor Bet?, Proc. IEEE Am. Control Conf. (ACC), Milwaukee, January 2018.

    Google Scholar 

  19. Norkin, B.V., On Numerical Solution of the Stochastic Optimal Control Problem for the Dividend Policy of an Insurance Company, Komp’yutern. Mat., 2014, no. 1, pp. 131–140.

    Google Scholar 

  20. Kan, Yu.S. and Kibzun, A.I., Zadachi stokhasticheskogo programmirovaniya s veroyatnostnymi kriteriyami (Stochastic Programming Problems with Probability Criteria), Moscow: Fizmatlit, 2009.

    MATH  Google Scholar 

  21. Kibzun, A.I. and Kan, Yu.S., Stochastic Programming Problems with Probability and Quantile Functions, Chichester: Wiley, 1996.

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Moscow Aviation Institute (National Research University), Moscow, Russia

    V. M. Azanov & Yu. S. Kan

Authors
  1. V. M. Azanov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu. S. Kan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to V. M. Azanov or Yu. S. Kan.

Additional information

Russian Text © V.M. Azanov, Yu.S. Kan, 2019, published in Avtomatika i Telemekhanika, 2019, No. 4, pp. 53–69.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azanov, V.M., Kan, Y.S. Refined Estimation of the Bellman Function for Stochastic Optimal Control Problems with Probabilistic Performance Criterion. Autom Remote Control 80, 634–647 (2019). https://doi.org/10.1134/S0005117919040039

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117919040039

Keywords

Search

Navigation