Robust Mean Field Games

Abstract

Recently there has been renewed interest in large-scale games in several research disciplines, with diverse application domains as in the smart grid, cloud computing, financial markets, biochemical reaction networks, transportation science, and molecular biology. Prior works have provided rich mathematical foundations and equilibrium concepts but relatively little in terms of robustness in the presence of uncertainties. In this paper, we study mean field games with uncertainty in both states and payoffs. We consider a population of players with individual states driven by a standard Brownian motion and a disturbance term. The contribution is threefold: First, we establish a mean field system for such robust games. Second, we apply the methodology to production of an exhaustible resource. Third, we show that the dimension of the mean field system can be significantly reduced by considering a functional of the first moment of the mean field process.

Appendix

In this appendix, we provide proofs for Propositions 2, 3, 4, 5, and 6, which constitute the main results of the paper.

1.1 Proof of Proposition 2

We utilize the strict convexity assumption for the function $c(\cdot )$ with respect to the control. Then the Hamiltonian is well posed, and the derivative of the Hamiltonian with respect to p provides the drift term of the state from which we deduce the (individual state) feedback optimal control.

1.2 Proof of Proposition 3

The first equation is a backward Hamilton–Jacobi–Bellman–Fleming equation starting at $T>0.$ The second equation is the evolution of the state distribution given by the forward equation, obtained from Definition 3. Collecting all, one arrives at the mean field system.

1.3 Proof of Proposition 4

The proof follows from Proposition 3 by letting $\sigma _t=0$ which eliminates the disturbance term.

1.4 Proof of Proposition 5

We first prove condition (33). To do this, let us write the Hamiltonian as:

$$\begin{aligned}&{H}_t(x_t,\partial _x v_t,\bar{m}_t)= \inf _{u} \Big \{-h(\bar{m}_t,\zeta _t^*)u \nonumber \\&\quad + \left[ \frac{a}{2} u^2 + b u\right] + \partial _x v_t (\alpha _t x_t+\beta _t u)\Big \}=0. \end{aligned}$$

(65)

Differentiating with respect to u, we obtain

$$\begin{aligned} a u - h(\bar{m}_t,\zeta _t^*) + b + \partial _x v_t \beta _t = 0 \end{aligned}$$

from which we can derive (33).

We now prove (29)–(32). First notice that (30)–(32) are the boundary conditions and derive straightforwardly from HJBF equations and the evolution of the distribution of states.

To prove (29), let us replace u appearing in the Hamiltonian (65) by its expression (33):

$$\begin{aligned} {H}_t(x_t,\partial _x v_t,\bar{m}_t)\,&=\,u_t^* [-h(\bar{m}_t,\zeta _t^*)+b + \partial _x v_t \beta _t] \\&\quad +\, \frac{a}{2} (u_t^*)^2 + \partial _x v_t \alpha _t x_t \\&= \left( h(\bar{m}_t,\zeta _t^*) - b - \partial _x v_t \beta _t \right) ^2 \left( -\frac{1}{a} + \frac{1}{2a} \right) + \partial _x v_t \alpha _t x_t\\&= -\frac{1}{2a} \left( h(\bar{m}_t,\zeta _t^*) - b - \partial _x v_t \beta _t \right) ^2 + \partial _x v_t \alpha _t x_t\\&=-\frac{1}{2a} \Big (h(\bar{m}_t,\zeta _t^*)^2 + b^2 + ( \partial _x v_t \beta _t)^2 - 2 h(\bar{m}_t,\zeta _t^*) b\\&\quad -\, 2 h(\bar{m}_t,\zeta _t^*) \partial _x v_t \beta _t + 2 b \partial _x v_t \beta _t \Big ) + \partial _x v_t \alpha _t x_t \\&= -\,\frac{1}{2a} \beta _t^2 |\partial _x v_t|^2 + \Big [ -\frac{1}{2a} (- 2 h(\bar{m}_t,\zeta _t^*) \beta _t + 2 b \beta _t) \\&\quad +\,\alpha _t x_t\Big ] \partial _x v_t -\frac{1}{2a} \left( h(\bar{m}_t,\zeta _t^*)^2 + b^2 - 2 h(\bar{m}_t,\zeta ^*_t)b \right) . \end{aligned}$$

Using the above expression of the Hamiltonian in the HJBF equation (21), we arrive at (29).

To prove (31), we simply plug (13) and (33) into (24), and this concludes the proof.

1.5 Proof of Proposition 6

To prove that the feedback policy $u^*$ as in (19) solves the disturbance attenuation problem (item 1), observe that

$$\begin{aligned} J^{\infty }(x,u^*,m^*,\zeta ):= & {} \mathbb {E} \Big (g(x_T) + \int _{0}^T c_t(x_t,u^*_t,m^*_t) \mathrm{d}t\Big ).\\= & {} \mathbb {E} \Big (v_T(x_T) +\int _{0}^T c_t(x_t,u^*_t,m^*_t) \mathrm{d}t\Big ).\\= & {} \mathbb {E} \Big ( v_0(x) + \int _{0}^T ( \partial _t v_t(x) + \partial _x v_t(x) \frac{\mathrm{d}x_t}{\mathrm{d}t} + c_t(x_t,u^*_t,m^*_t) )\mathrm{d}t\Big ).\\= & {} \mathbb {E} \Big ( v_0(x) + \int _{0}^T ( - \tilde{H}_t(x,\partial _xv_t,m^*_t)-\frac{\sigma _t^2 x^2}{2}\partial ^2_{xx}v_t(x) \\&\quad +\,\partial _x v_t(x) \frac{\mathrm{d}x_t}{\mathrm{d}t} + c_t(x_t,u^*_t,m^*_t) )\hbox {d}t\Big )\\\le & {} \mathbb {E} \Big ( v_0(x) + \int _{0}^T ( - c(x,u^*_t,m^*_t) +\gamma ^2(\zeta ^*)^2 - \partial _x v_t(x) \frac{\mathrm{d}x_t^*}{\mathrm{d}t}\\&-\,\frac{\sigma _t^2 x^2}{2}\partial ^2_{xx}v_t(x) + \partial _x v_t(x) \frac{\mathrm{d}x_t^*}{\mathrm{d}t} + c_t(x_t,u^*_t,m^*_t) )\mathrm{d}t\Big ) \\&\quad +\,\gamma ^2 q_0(x) - \gamma ^2 q_0(x) \\\le & {} \mathbb {E} \Big ( \int _{0}^T ( \gamma ^2(\zeta ^*)^2 - \frac{\sigma _t^2 x^2}{2}\partial ^2_{xx}v_t(x) )\mathrm{d}t\Big ) + \gamma ^2 q_0(x)\\\le & {} \gamma ^2 \left( \Vert \zeta \Vert ^2 + q_0(x)\right) . \end{aligned}$$

The first equality is by definition itself of cost $J^{\infty }(x,u^*,m^*,\zeta )$. In the second equality, we use the boundary condition (22). In the third equality, we replace $v_T(x_T)$ by $v_0(x) + \int _{0}^T ( \partial _t v_t(x) \hbox {d}t + \partial _x v_t(x) \hbox {d}x_t$. In the fourth equality, we use the HJBF equation (15). In the fifth inequality, we use the expression of the robust Hamiltonian (17). Here $\mathrm{d}x_t^*$ denotes the infinitesimal state under $u^*$ and $\zeta ^*$.

To prove that $\mathbb R^{|\mathcal {X}|}_-$ is attainable by $\mathcal {V}_0(x)$ (item 2.), we need to show that $\lim _{t \rightarrow 0} \Vert [ \mathcal {V}_t ]_+ \Vert = 0$. To see that this holds true, note that for every $x \in \mathcal {X},\,v_0(x) \le \gamma ^2 q_0(x)$ as $\gamma \ge \hat{\gamma }^{\mathrm{NCL}}$.