Skip to main content
Log in

Numerical Solutions for Optimal Control of Stochastic Kolmogorov Systems

  • Published:
Journal of Systems Science and Complexity Aims and scope Submit manuscript

Abstract

This work is concerned with controlled stochastic Kolmogorov systems. Such systems have received much attention recently owing to the wide range of applications in biology and ecology. Starting with the basic premise that the underlying system has an optimal control, this paper is devoted to designing numerical methods for approximation. Different from the existing literature on numerical methods for stochastic controls, the Kolmogorov systems take values in the first quadrant. That is, each component of the state is nonnegative. The work is designing an appropriate discrete-time controlled Markov chain to be in line with (locally consistent) the controlled diffusion. The authors demonstrate that the Kushner and Dupuis Markov chain approximation method still works. Convergence of the numerical scheme is proved under suitable conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Guo L and Chen H F, The Aström-Wittenmark self-tuning regulator revisited and ELS-based adaptive trackers, IEEE Trans. Automat. Control, 1991, 36: 802–812.

    Article  MathSciNet  Google Scholar 

  2. Guo L, Exploring the capability and limits of the feedback mechanism, Proc. Internat. Congress Mathematicians, 2002, III: 785–794, Higher Ed. Press, Beijing.

    MathSciNet  MATH  Google Scholar 

  3. Hofbauer J and Sigmund K, Evolutionary Games and Population Dynamics, Cambridge University Press, Cambridge, 1998.

    Book  Google Scholar 

  4. Benaïm M and Lobry C, Lotka-Volterra with randomly fluctuating environments or “how switching between beneficial environments can make survival harder”, Ann. Appl. Probab., 2016, 26: 3754–3785.

    Article  MathSciNet  Google Scholar 

  5. Benaïm M, Stochastic persistence, 2018, preprint.

  6. Hening A and Nguyen D H, Coexistence and extinction for stochastic Kolmogorov systems, Ann. Appl. Probab., 2018, 28: 1893–1942.

    Article  MathSciNet  Google Scholar 

  7. Dang H, Du D, and Yin G, Existence of stationary distributions for Kolmogorov systems of competitive type under telegraph noise, J. Differential Eqs., 2014, 257: 2078–2101.

    Article  MathSciNet  Google Scholar 

  8. Tran K and Yin G, Optimal harvesting strategies for stochastic competitive Lotka-Volterra ecosystems, Automatica, 2015, 55: 236–246.

    Article  MathSciNet  Google Scholar 

  9. Du N, Nguyen D, and Yin G, Conditions for permanence and ergodicity of certain stochastic predator-prey models, J. Appl. Probab., 2016, 53: 187–202.

    Article  MathSciNet  Google Scholar 

  10. Tran K and Yin G, Numerical methods for optimal harvesting strategies in random environments under partial observations, Automatica, 2016, 70: 74–85.

    Article  MathSciNet  Google Scholar 

  11. Dieu N, Du D, Nguyen D, et al., Protection zones for survival of species in random environment, SIAM J. Appl. Math., 2016, 76: 1382–1402.

    Article  MathSciNet  Google Scholar 

  12. Nguyen D and Yin G, Modeling and analysis of switching diffusion systems: Past dependent switching with a countable state space, SIAM J. Control Optim., 2016, 54: 2450–2477.

    Article  MathSciNet  Google Scholar 

  13. Nguyen D and Yin G, Coexistence and exclusion of stochastic competitive Lotka-Volterra models, J. Differential Eqs., 2017, 262: 1192–1225.

    Article  MathSciNet  Google Scholar 

  14. Hening A, Nguyen D, and Yin G, Stochastic population growth in spatially heterogeneous environments: The density-dependent case, J. Math. Bio., 2018, 76: 697–754.

    Article  MathSciNet  Google Scholar 

  15. Hening A, Tran K, Phan T, et al., Harvesting of interacting stochastic populations, J. Math. Bio., 2019, 79: 533–570.

    Article  MathSciNet  Google Scholar 

  16. Yin G and Wen Z, Stochastic Kolmogorov systems driven by wideband noises, Physica A, 2019, 531: 121746.

    Article  MathSciNet  Google Scholar 

  17. Nguyen D, Nguyen N, and Yin G, General nonlinear stochastic systems motivated by chemostat models: Complete characterization of long-time behavior, optimal controls, and applications to wastewater treatment, Stochastic Process Appl., 2020, 130: 4608–4642.

    Article  MathSciNet  Google Scholar 

  18. Nguyen D and Yin G, Sustainable harvesting policies under long-run average criteria: Near optimality, Appl. Math. Optim., 2020, 20: 443–478.

    Article  MathSciNet  Google Scholar 

  19. Nguyen D, Nguyen N, and Yin G, Analysis of a spatially inhomogeneous stochastic partial differential equation epidemic model, J. Appl. Probab., 2020, 57: 613–636.

    Article  MathSciNet  Google Scholar 

  20. Nguyen D, Yin G, and Zhu C, Long-term analysis of a stochastic SIRS model with general incidence rates, SIAM J. Appl. Math., 2020, 80: 814–838.

    Article  MathSciNet  Google Scholar 

  21. Yin G and Wen Z, Two-time-scale regime-switching stochastic Kolmogorov systems with wideband noises, Ann. Acad. Rom. Sci. Ser. Math. Appl., 2020, 12: 62–89.

    MathSciNet  MATH  Google Scholar 

  22. Tran K and Yin G, Optimal control and numerical methods for hybrid stochastic SIS models, Nonlinear Ana.: Hybrid Sys., 2021, 41: 101051.

    MathSciNet  Google Scholar 

  23. Kermack W O and McKendrick A G, A contribution to the mathematical theory of epidemics, Proceedings of Royal Society of London, Series A, 1927, 115: 700–721.

    MATH  Google Scholar 

  24. Kushner H and Dupuis P, Numerical Methods for Stochstic Control Problems in Continuous Time, 2nd Edition, Springer-Verlag, New York, 2001.

    Book  Google Scholar 

  25. Kushner H, Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory, MIT Press, Cambridge, MA, 1984.

    MATH  Google Scholar 

  26. Kushner H, Numerical methods for stochastic control problems in continuous time, SIAM J. Control Optim., 1990, 28: 999–1048.

    Article  MathSciNet  Google Scholar 

  27. Song Q S and Yin G, Rates of convergence of numerical methods for controlled regime-switching diffusions with stopping times in the costs, SIAM J. Control Optim., 2009, 48: 1831–1857.

    Article  MathSciNet  Google Scholar 

  28. Bayraktar E, Song Q, and Yang J, On the continuity of stochastic exit time control problems, Stoch. Anal. Appl., 2011, 29: 48–60.

    Article  MathSciNet  Google Scholar 

  29. Kushner H and Yin G, Stochastic Approximation and Recursive Algorithms and Applications, 2nd Edition, Springer-Verlag, New York, 2003.

    MATH  Google Scholar 

  30. Krause A, Kurowski L, Yawar K, et al., Stochastic epidemic metapopulation models on networks: SIS dynamics and control strategies, J. Theoret. Biol., 2018, 449: 35–52.

    Article  MathSciNet  Google Scholar 

  31. Han J, Jentzen A, and E W, Solving high-dimensional partial differential equations using deep learning, Proc. National Acad. Sci., 2018, 115(34): 8505–8510.

    Article  MathSciNet  Google Scholar 

  32. Qiu W, Song Q, and Yin G, Solving elliptic Hamilton-Jacobi-Bellman equations in a value space, IEEE Control Sys. Lett., 2011, 5: 55–60.

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to George Yin, Zhexin Wen, Hongjiang Qian or Huy Nguyen.

Additional information

The research of G. Yin and H. Qian was supported in part by the Air Force Office of Scientific Research, and the research of Z. Wen was supported in part by Postdoctoral Foundation of Central South University.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yin, G., Wen, Z., Qian, H. et al. Numerical Solutions for Optimal Control of Stochastic Kolmogorov Systems. J Syst Sci Complex 34, 1703–1722 (2021). https://doi.org/10.1007/s11424-021-1170-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11424-021-1170-5

Keywords

Navigation