Abstract
This paper reviews the mean field social (MFS) optimal control problem for multi-agent dynamic systems and the mean-field-type (MFT) optimal control problem for single-agent dynamic systems within the linear quadratic (LQ) framework. For the MFS control problem, this review discusses the existing conclusions on optimization in dynamic systems affected by both additive and multiplicative noises. In exploring MFT optimization, the authors first revisit researches associated with single-player systems constrained by these dynamics. The authors then extend the proposed review to scenarios that include multiple players engaged in Nash games, Stackelberg games, and cooperative Pareto games. Finally, the paper concludes by emphasizing future research on intelligent algorithms for mean field optimization, particularly using reinforcement learning method to design strategies for models with unknown parameters.
References
Lasry J M and Lions P L, Mean field games, Japanese Journal of Mathematics, 2007, 2): 229–260.
Huang M, Large-population LQG games involving a major player: The Nash certainty equivalence principle, SIAM Journal on Control and Optimization, 2010, 48(5): 3318–3353.
Huang M, Caines P E, and Malhamé R P, Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized ε-nash equilibria, IEEE Transactions on Automatic Control, 2007, 52(9): 1560–1571.
Huang M, Caines P E, and Malhamé R P, Social optima in mean field LQG control: Centralized and decentralized strategies, IEEE Transactions on Automatic Control, 2012, 57(7): 1736–1751.
Andersson D and Djehiche B, A maximum principle for SDE’s of mean-field type, Applied Mathematics & Optimization, 2011, 63(3): 341–356.
Yong J, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 2013, 51(4): 2809–2838.
Elliott R, Li X, and Ni Y, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 2013, 49(11): 3222–3233.
Kolokoltsov V N, Li J, and Yang W, Mean field games and nonlinear Markov processes, 2011, arXiv: 1112.3744.
Moon J and Basar T, Linear quadratic risk-sensitive and robust mean field games, IEEE Transactions on Automatic Control, 2016, 62(3): 1062–1077.
Tembine H, Zhu Q, and Basar T, Risk-sensitive mean-field stochastic differential games, Proceedings of the 18th IFAC World Congress, 2011, 3222–3227.
Hafayed M, A mean-field necessary and sufficient conditions for optimal singular stochastic control, Communications in Mathematical Statistics, 2013, 1(4): 417–435.
Jiang X, Wang Y, Zhao D, et al., Online Pareto optimal control of mean-field stochastic multiplayer systems using policy iteration, Science China Information Sciences, 2024, 67(4): 140202.
Ni Y, Li X, and Zhang J, Indefinite mean-field stochastic linear-quadratic optimal control: From finite horizon to infinite horizon, IEEE Transactions on Automatic Control, 2015, 61(11): 3269–3284.
Yang C, Li J, Semasinghe P, et al., Distributed interference and energy-aware power control for ultra-dense D2D networks: A mean field game, IEEE Transactions on Wireless Communications, 2017, 16(2): 1205–1217.
Couillet R, Perlaza S M, Tembine H, et al., Electric vehicles in the smart grid: A mean field game analysis, IEEE Journal on Selected Areas in Communications, 2012, 30(6): 1086–1096.
Ma Z, Callaway D S, and Hiskens I A, Decentralized charging control for large populations of plug-in electric vehicles, IEEE Transactions on Control Systems Technology, 2011, 21(1): 67–78.
Lin R, Xu Z, Huang X, et al., Optimal scheduling management of the parking lot and decentralized charging of electric vehicles based on mean field game, Applied Energy, 2022, 328): 120198.
Banez R A, Tembine H, Li L, et al., Mean-field-type game-based computation offloading in multi-access edge computing networks, IEEE Transactions on Wireless Communications, 2020, 19(12): 8366–8381.
Wang Y, Yu F R, Tang H, et al., A mean field game theoretic approach for security enhancements in mobile ad hoc networks, IEEE Transactions on Wireless Communications, 2014, 13(3): 1616–1627.
Ge X, Jia H, Zhong Y, et al., Energy efficient optimization of wireless-powered 5G full duplex cellular networks: A mean field game approach, IEEE Transactions on Green Communications and Networking, 2019, 3(2): 455–467.
Guo J, Guo Q, Mou C, et al., A mean field game model of staking system, Digital Finance, 2024, DOI: https://doi.org/10.1007/s42521024001134.
Wang B and Huang M, Mean field production output control with sticky prices: Nash and social solutions, Automatica, 2019, 100): 90–98.
Bauso D, Tembine H, and Basar T, Opinion dynamics in social networks through mean-field games, SIAM Journal on Control and Optimization, 2016, 54(6): 3225–3257.
Bauso D, Zhang X, and Papachristodoulou A, Density flow in dynamical networks via mean-field games, IEEE Transactions on Automatic Control, 2016, 62(3): 1342–1355.
Bauch C T and Earn D J, Vaccination and the theory of games, Proceedings of the National Academy of Sciences, 2004, 101(36): 13391–13394.
Carmona R and Delarue F, Probabilistic analysis of mean-field games, SIAM Journal on Control and Optimization, 2013, 51(4): 2705–2734.
Dong B, Nie T, and Wu Z, Maximum principle for discrete-time stochastic control problem of mean-field type, Automatica, 2022, 144): 110497.
Nuño G and Moll B, Social optima in economies with heterogeneous agents, Review of Economic Dynamics, 2018, 28): 150–180.
Wang G and Wu Z, A maximum principle for mean-field stochastic control system with noisy observation, Automatica, 2022, 137): 110135.
Huang M, Li T, and Zhang J, Stochastic approximation based consensus dynamics over Markovian networks, SIAM Journal on Control and Optimization, 2015, 53(6): 3339–3363.
Moon J, Risk-sensitive maximum principle for stochastic optimal control of mean-field type Markov regime-switching jump-diffusion systems, International Journal of Robust and Nonlinear Control, 2021, 31(6): 2141–2167.
Weintraub G Y, Benkard C L, and Van Roy B, Markov perfect industry dynamics with many firms, Econometrica, 2008, 76(6): 1375–1411.
Bensoussan A, Frehse J, and Yam P, Mean Field Games and Mean Field Type Control Theory, Springer, New York, 2013.
Caines P E, Huang M, and Malhame R P, Mean field games, Handbook of Dynamic Game Theory, Eds. by Basar T and Zaccour G, Springer, Berlin, 2017.
Carmona R and Delarue F, Probabilistic Theory of Mean Field Games with Applications, III, Springer Nature, Berlin, 2018.
Gomes D A and Saude J, Mean field games models — A brief survey, Dynamic Games and Applications, 2014, 4(2): 110–154.
Guéant O, Lasry J M, and Lions P L, Mean field games and applications, Paris-Princeton Lectures on Mathematical Finance, Springer, Berlin, 2011.
Ho Y C, Team decision theory and information structures, Proceedings of the IEEE, 1980, 68(6): 644–654.
Radner R, Team decision problems, Annals of Mathematical Statistics, 1962, 33(3): 857–881.
Wang B, Zhang H, and Zhang J, Mean field linear-quadratic control: Uniform stabilization and social optimality, Automatica, 2020, 121): 109088.
Chen X and Huang M, Linear-quadratic mean field control: The invariant subspace method, Automatica, 2019, 107): 582–586.
Wang B and Zhang J, Mean field games for large-population multiagent systems with Markov jump parameters, SIAM Journal on Control and Optimization, 2012, 50(4): 2308–2334.
Li T and Zhang J, Decentralized tracking-type games for multi-agent systems with coupled ARX models: Asymptotic Nash equilibria, Automatica, 2008, 44(3): 713–725.
Wang B, Huang J, and Zhang J, Social optima in robust mean field LQG control: From finite to infinite horizon, IEEE Transactions on Automatic Control, 2020, 66(4): 1529–1544.
Wang B and Liang Y, Robust mean field social control problems with applications in analysis of opinion dynamics, International Journal of Control, 2022, 95(12): 3309–3325.
Wang B and Zhang J, Social optima in mean field linear-quadratic-Gaussian models with Markov jump parameters, SIAM Journal on Control and Optimization, 2017, 55(1): 429–456.
Huang M and Nguyen S L, Linear-quadratic mean field social optimization with a major player, 2019, arXiv: 1904.03346.
Arabneydi J and Aghdam A G, Deep teams: Decentralized decision making with finite and infinite number of agents, IEEE Transactions on Automatic Control, 2020, 65(10): 4230–4245.
Arabneydi J and Mahajan A, Linear quadratic mean field teams: Optimal and approximately optimal decentralized solutions, 2016, arXiv: 1609.00056.
Sen N, Huang M, and Malhamé R P, Mean field social control with decentralized strategies and optimality characterization, 2016 IEEE 55th Conference on Decision and Control, 2016, 6056–6061.
Salhab R, Le Ny J, and Malhamé R P, Dynamic collective choice: Social optima, IEEE Transactions on Automatic Control, 2018, 63(10): 3487–3494.
Albi G, Choi Y P, Fornasier M, et al., Mean field control hierarchy, Applied Mathematics & Optimization, 2017, 76(1): 93–135.
Fornasier M and Solombrino F, Mean-field optimal control, ESAIM: Control, Optimisation and Calculus of Variations, 2014, 20(4): 1123–1152.
Kac M, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume III. University of California Press, California, 1956.
McKean Jr H P, A class of Markov processes associated with nonlinear parabolic equations, Proceedings of the National Academy of Sciences of the USA, 1966, 56(6): 1907–1911.
Ahmed N U and Ding X, Controlled McKean-Vlasov equations, Communications in Applied Analysis, 2001, 5(2): 183–206.
Buckdahn R, Li J, and Peng S, Mean-field backward stochastic differential equations and related partial differential equations, Stochastic Processes and Applications, 2009, 119(10): 3133–3154.
Crisan D and Xiong J, Approximate McKean-Vlasov representations for a class of SPDEs, Stochastics, 2010, 82(1): 53–68.
Dawson D A, Critical dynamics and fluctuations for a mean-field model of cooperative behavior, Journal of Statistical Physics, 1983, 31): 29–85.
Graham C, McKean-Vlasov Itô-Skorohod equations, and nonlinear diffusions with discrete jump sets, Stochastic Processes and Applications, 1992, 40(1): 69–82.
Meyer-Brandis T, Øksendal B, and Zhou X, A mean-field stochastic maximum principle via Malliavin calculus, Stochastics, 2012, 84): 643–666.
Huang J, Li X, and Yong J, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control and Related Fields, 2015, 5(1): 97–139.
Ni Y, Elliott R, and Li X, Discrete-time mean-field stochastic linear-quadratic optimal control problems, II: Infinite horizon case, Automatica, 2015, 57): 65–77.
Huang J, Wang S, and Wu Z, Backward mean-field linear-quadratic-Gaussian (LQG) games: Full and partial information, IEEE Transactions on Automatic Control, 2016, 61(12): 3784–3796.
Wang G, Zhang C, and Zhang W, Stochastic maximum principle for mean-field type optimal control under partial information, IEEE Transactions on Automatic Control, 2013, 59(2): 522–528.
Hafayed M and Abbas S, On near-optimal mean-field stochastic singular controls: Necessary and sufficient conditions for near-optimality, Journal of Optimization Theory and Applications, 2014, 160): 778–808.
Yong J, Linear-quadratic optimal control problems for mean-field stochastic differential equations: Time consistent solutions, Transactions of the American Mathematical Society, 2017, 369(8): 5467–5523.
Barreiro-Gomez J, Duncan T E, and Tembine H, Linear-quadratic mean-field-type games: Jump-diffusion process with regime switching, IEEE Transactions on Automatic Control, 2019, 64(10): 4329–4336.
Graber P J, Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource, Applied Mathematics and Optimization, 2016, 74): 459–486.
Qi Q, Xie L, and Zhang H, Linear-quadratic optimal control for discrete-time mean-field systems with input delay, IEEE Transactions on Automatic Control, 2021, 67(8): 4161–4167.
Zhang W, Xie L, and Chen B S, Stochastic H2/H∞Control: A Nash Game Approach, CRC Press, Florida, 2017.
Zames G, Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverse, IEEE Transactions on Automatic Control, 1981, 26(2): 301–320.
Hinrichsen D and Pritchard A J, Stochastic H∞, SIAM Journal on Control and Optimization, 1998, 36(5): 1504–1538.
Zhang W, Zhong S, and Jiang X, Finite-time annular domain stability and asynchronous H∞ control for stochastic switching Markov jump systems, IEEE Transactions on Automatic Control, 2024, DOI: https://doi.org/10.1109/TAC.2024.3379248.
Chen B S and Zhang W, Stochastic H2/H∞ control with state-dependent noise, IEEE Transactions on Automatic Control, 2004, 49(1): 45–57.
Hou T, Zhang W, and Ma H, Finite horizon H2/H∞ control for discrete-time stochastic systems with Markovian jumps and multiplicative noise, IEEE Transactions on Automatic Control, 2010, 55(5): 1185–1191.
Ma H, Mou C, and Ho D W, Open-loop H2/H∞ control for discrete-time mean-field stochastic systems, IEEE Transactions on Automatic Control, 2024, DOI: https://doi.org/10.1109/TAC.2024.3395018.
Ma H, Zhang W, and Hou T, Infinite horizon H2/H∞ control for discrete-time time-varying Markovian jump systems with multiplicative noise, Automatica, 2012, 48(7): 1447–1454.
Ma L and Zhang W, Output feedback H∞ control for discrete-time mean-field stochastic systems, Asian Journal of Control, 2015, 17(6): 2241–2251.
Ma L, Zhang T, Zhang W, et al., Finite horizon mean-field stochastic H2/H∞ control for continuous-time systems with (x, v)-dependent noise, Journal of the Franklin Institute, 2015, 352): 5393–5414.
Zhang W, Huang Y, and Zhang H, Stochastic H2/H∞ control for discrete-time systems with state and disturbance dependent noise, Automatica, 2007, 43(3): 513–521.
Zhang W, Huang Y, and Xie L, Infinite horizon stochastic H2/H∞ control for discrete-time systems with state and disturbance dependent noise, Automatica, 2008, 44(9): 2306–2316.
Abou-Kandil H and Bertrand P, Analytical solution for an open-loop Stackelberg game, IEEE Transactions on Automatic Control, 1985, 30(12): 1222–1224.
Yong J, A leader-follower stochastic linear quadratic differential game, SIAM Journal on Control and Optimization, 2002, 41(4): 1015–1041.
Mukaidani H and Xu H, Infinite horizon linear-quadratic Stackelberg games for discrete-time stochastic systems, Automatica, 2017, 76): 301–308.
Xu J and Zhang H, Sufficient and necessary open-loop Stackelberg strategy for two-player game with time delay, IEEE Transactions on Cybernetics, 2015, 46(2): 438–449.
Moon J, Linear-quadratic stochastic Stackelberg differential games for jump-diffusion systems, SIAM Journal on Control and Optimization, 2021, 59(2): 954–976.
Aberkane S and Dragan V, An addendum to the problem of zero-sum LQ stochastic mean-field dynamic games, Automatica, 2023, 153): 111007.
Lin Y, Jiang X, and Zhang W, Necessary and sufficient conditions for Pareto optimality of the stochastic systems in finite horizon, Automatica, 2018, 94): 341–348.
Lv S, Xiong J, and Zhang X, Linear quadratic leader-follower stochastic differential games for mean-field switching diffusions, Automatica, 2023, 154): 111072.
Moon J and Yang H J, Linear-quadratic time-inconsistent mean-field type Stackelberg differential games: Time-consistent open-loop solutions, IEEE Transactions on Automatic Control, 2020, 66(1): 375–382.
Engwerda J, LQ Dynamic Optimization and Differential Games, John Wiley and Sons, Chichester, 2005.
Engwerda J, The regular convex cooperative linear quadratic control problem, Automatica, 2008, 44(9): 2453–2457.
Engwerda J, Necessary and sufficient conditions for Pareto optimal solutions of cooperative differential games, SIAM Journal on Control and Optimization, 2010, 48(6): 3859–3881.
Jiang X, Su S, and Zhao D, Pareto optimal strategy under H∞ constraint for the mean-field stochastic systems in infinite horizon, IEEE Transactions on Cybernetics, 2022, 53(11): 6963–6976.
Lin Y, Necessary/sufficient conditions for Pareto optimality in finite horizon mean-field type stochastic differential game, Automatica, 2020, 119): 108951.
Lin Y, Jiang X, and Zhang W, An open-loop Stackelberg strategy for the linear quadratic mean-field stochastic differential game, IEEE Transactions on Automatic Control, 2019, 64(1): 97–110.
Lin Y, Zhang T, and Zhang W, Pareto-based guaranteed cost control of the uncertain mean-field stochastic systems in infinite horizon, Automatica, 2018, 92): 197–209.
Lin Y and Zhang W, Necessary/sufficient conditions for Pareto optimum in cooperative difference game, Optimal Control Applications and Methods, 2018, 39(2): 1043–1060.
Zhang W, Peng C, and Jiang X, Pareto stochastic cooperative games in multiobjective dynamic optimization problems: A survey, Control and Decision, 2023, 38(7): 1789–1801.
Zhang W and Chen B S, On stabilizability and exact observability of stochastic systems with their applications, Automatica, 2004, 40(1): 87–94.
Zhang W, Zhang H, and Chen B S, Generalized Lyapunov equation approach to state-dependent stochastic stabilization/detectability criterion, IEEE Transactions on Automatic Control, 2008, 53(7): 1630–1642.
Wang B and Zhang H, Indefinite linear quadratic mean field social control problems with multiplicative noise, IEEE Transactions on Automatic Control, 2021, 66(11): 5221–5236.
Zhang W, Ma L, and Zhang T, Discrete-time mean-field stochastic H2/H∞ control, Journal of Systems Science & Complexity, 2017, 30(3): 765–781.
Peng C and Zhang W, Pareto optimality in infinite horizon mean-field stochastic cooperative linear-quadratic difference games, IEEE Transactions on Automatic Control, 2023, 68(7): 4113–4126.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors declare no conflict of interest.
Additional information
This research was supported by the National Natural Science Foundation of China under Grant Nos. 62103442, 12326343, 62373229, the Research Grants Council of the Hong Kong Special Administrative Region, China under Grant Nos. CityU 11213023, 11205724, the Natural Science Foundation of Shandong Province under Grant No. ZR2021QF080, the Taishan Scholar Project of Shandong Province under Grant No. tsqn202408110, the Fundamental Research Foundation of the Central Universities under Grant No. 23CX06024A, and the Outstanding Youth Innovation Team in Shandong Higher Education Institutions under Grant No. 2023KJ061.
Rights and permissions
About this article
Cite this article
Jiang, X., Ho, D.W.C. & Zhang, W. A Brief Review on Mean Field Optimal Control Problem from a Linear Quadratic Perspective. J Syst Sci Complex 38, 390–420 (2025). https://doi.org/10.1007/s11424-025-4501-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11424-025-4501-0