Abstract
This work is concerned with controlled stochastic Kolmogorov systems. Such systems have received much attention recently owing to the wide range of applications in biology and ecology. Starting with the basic premise that the underlying system has an optimal control, this paper is devoted to designing numerical methods for approximation. Different from the existing literature on numerical methods for stochastic controls, the Kolmogorov systems take values in the first quadrant. That is, each component of the state is nonnegative. The work is designing an appropriate discrete-time controlled Markov chain to be in line with (locally consistent) the controlled diffusion. The authors demonstrate that the Kushner and Dupuis Markov chain approximation method still works. Convergence of the numerical scheme is proved under suitable conditions.
Similar content being viewed by others
References
Guo L and Chen H F, The Aström-Wittenmark self-tuning regulator revisited and ELS-based adaptive trackers, IEEE Trans. Automat. Control, 1991, 36: 802–812.
Guo L, Exploring the capability and limits of the feedback mechanism, Proc. Internat. Congress Mathematicians, 2002, III: 785–794, Higher Ed. Press, Beijing.
Hofbauer J and Sigmund K, Evolutionary Games and Population Dynamics, Cambridge University Press, Cambridge, 1998.
Benaïm M and Lobry C, Lotka-Volterra with randomly fluctuating environments or “how switching between beneficial environments can make survival harder”, Ann. Appl. Probab., 2016, 26: 3754–3785.
Benaïm M, Stochastic persistence, 2018, preprint.
Hening A and Nguyen D H, Coexistence and extinction for stochastic Kolmogorov systems, Ann. Appl. Probab., 2018, 28: 1893–1942.
Dang H, Du D, and Yin G, Existence of stationary distributions for Kolmogorov systems of competitive type under telegraph noise, J. Differential Eqs., 2014, 257: 2078–2101.
Tran K and Yin G, Optimal harvesting strategies for stochastic competitive Lotka-Volterra ecosystems, Automatica, 2015, 55: 236–246.
Du N, Nguyen D, and Yin G, Conditions for permanence and ergodicity of certain stochastic predator-prey models, J. Appl. Probab., 2016, 53: 187–202.
Tran K and Yin G, Numerical methods for optimal harvesting strategies in random environments under partial observations, Automatica, 2016, 70: 74–85.
Dieu N, Du D, Nguyen D, et al., Protection zones for survival of species in random environment, SIAM J. Appl. Math., 2016, 76: 1382–1402.
Nguyen D and Yin G, Modeling and analysis of switching diffusion systems: Past dependent switching with a countable state space, SIAM J. Control Optim., 2016, 54: 2450–2477.
Nguyen D and Yin G, Coexistence and exclusion of stochastic competitive Lotka-Volterra models, J. Differential Eqs., 2017, 262: 1192–1225.
Hening A, Nguyen D, and Yin G, Stochastic population growth in spatially heterogeneous environments: The density-dependent case, J. Math. Bio., 2018, 76: 697–754.
Hening A, Tran K, Phan T, et al., Harvesting of interacting stochastic populations, J. Math. Bio., 2019, 79: 533–570.
Yin G and Wen Z, Stochastic Kolmogorov systems driven by wideband noises, Physica A, 2019, 531: 121746.
Nguyen D, Nguyen N, and Yin G, General nonlinear stochastic systems motivated by chemostat models: Complete characterization of long-time behavior, optimal controls, and applications to wastewater treatment, Stochastic Process Appl., 2020, 130: 4608–4642.
Nguyen D and Yin G, Sustainable harvesting policies under long-run average criteria: Near optimality, Appl. Math. Optim., 2020, 20: 443–478.
Nguyen D, Nguyen N, and Yin G, Analysis of a spatially inhomogeneous stochastic partial differential equation epidemic model, J. Appl. Probab., 2020, 57: 613–636.
Nguyen D, Yin G, and Zhu C, Long-term analysis of a stochastic SIRS model with general incidence rates, SIAM J. Appl. Math., 2020, 80: 814–838.
Yin G and Wen Z, Two-time-scale regime-switching stochastic Kolmogorov systems with wideband noises, Ann. Acad. Rom. Sci. Ser. Math. Appl., 2020, 12: 62–89.
Tran K and Yin G, Optimal control and numerical methods for hybrid stochastic SIS models, Nonlinear Ana.: Hybrid Sys., 2021, 41: 101051.
Kermack W O and McKendrick A G, A contribution to the mathematical theory of epidemics, Proceedings of Royal Society of London, Series A, 1927, 115: 700–721.
Kushner H and Dupuis P, Numerical Methods for Stochstic Control Problems in Continuous Time, 2nd Edition, Springer-Verlag, New York, 2001.
Kushner H, Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory, MIT Press, Cambridge, MA, 1984.
Kushner H, Numerical methods for stochastic control problems in continuous time, SIAM J. Control Optim., 1990, 28: 999–1048.
Song Q S and Yin G, Rates of convergence of numerical methods for controlled regime-switching diffusions with stopping times in the costs, SIAM J. Control Optim., 2009, 48: 1831–1857.
Bayraktar E, Song Q, and Yang J, On the continuity of stochastic exit time control problems, Stoch. Anal. Appl., 2011, 29: 48–60.
Kushner H and Yin G, Stochastic Approximation and Recursive Algorithms and Applications, 2nd Edition, Springer-Verlag, New York, 2003.
Krause A, Kurowski L, Yawar K, et al., Stochastic epidemic metapopulation models on networks: SIS dynamics and control strategies, J. Theoret. Biol., 2018, 449: 35–52.
Han J, Jentzen A, and E W, Solving high-dimensional partial differential equations using deep learning, Proc. National Acad. Sci., 2018, 115(34): 8505–8510.
Qiu W, Song Q, and Yin G, Solving elliptic Hamilton-Jacobi-Bellman equations in a value space, IEEE Control Sys. Lett., 2011, 5: 55–60.
Author information
Authors and Affiliations
Corresponding authors
Additional information
The research of G. Yin and H. Qian was supported in part by the Air Force Office of Scientific Research, and the research of Z. Wen was supported in part by Postdoctoral Foundation of Central South University.
Rights and permissions
About this article
Cite this article
Yin, G., Wen, Z., Qian, H. et al. Numerical Solutions for Optimal Control of Stochastic Kolmogorov Systems. J Syst Sci Complex 34, 1703–1722 (2021). https://doi.org/10.1007/s11424-021-1170-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-021-1170-5