Appendix A: Proof of Theorem 2.1
To prove Theorem 2.1, we adopt the technique of backward induction. Consider first the last operational stage \( T \); invoking Theorem 2.1, we have
$$ W(T,x;u_{T - } ) = \mathop {\hbox{max} }\limits_{{u_{T}^{{}} }} \left\{ {g_{T} \left( {x,u_{T} ;u_{T - } } \right) + q_{T + 1} \left[ {f_{T} (x,u_{T} ;u_{T - } );u_{(T + 1) - } } \right]} \right\}. $$
(A.1)
The maximization operator in stage \( T \) involves \( u_{T}^{{}} \) only. However, the current state \( x \) and the previously executed controls \( u_{T - }^{{}} \) appear in the stage \( T \) maximization problem as given parameters. If the first-order conditions of the maximization problem in (A.1) satisfy the implicit function theorem, one can obtain the optimal controls \( u_{T}^{{}} \) as functions of \( x \) and \( u_{T - }^{{}} \). Substituting these optimal controls into the function on the RHS of (A.1) yields the function \( W(T,x;u_{T - }^{{}} ) \), which satisfies the optimal conditions of a maximum for given \( x \) and \( u_{T - }^{{}} \).
Consider the second last operational stage \( T - 1 \); invoking Theorem 2.1, we have
$$ \begin{aligned} W\left( {T - 1,x;u_{(T - 1) - } } \right) & = \mathop {\hbox{max} }\limits_{{u_{T - 1} }} \left\{ {g_{T - 1} \left( {x,u_{T - 1} ;u_{(T - 1) - } } \right)} \right. \\ & \quad + \;\left. {W\left[ {T,f_{T - 1} \left( {x,u_{T - 1} ;u_{(T - 1) - } } \right);u_{T - } } \right]} \right\}. \\ \end{aligned} $$
(A.2)
The maximization operator in stage \( T - 1 \) involves \( u_{T - 1}^{{}} \). The current state \( x \) and the previously executed controls \( u_{(T - 1) - }^{{}} \) appear in the stage \( T - 1 \) maximization problem as given parameters. If the first-order conditions of the maximization problem in (A.2) satisfy the implicit function theorem, one can obtain the optimal controls \( u_{T - 1}^{{}} \) as functions of \( x \) and previously determined controls \( u_{(T - 1) - }^{{}} \). Substituting these optimal controls into the function on the RHS of (A.2) yields the function \( W(T - 1,x;u_{(T - 1) - }^{{}} ) \).
Now consider stage \( k \in \{ T - 2,T - 3, \cdots ,2,1\} \); invoking Theorem 2.1, we have
$$ W(k,x;u_{k - } ) = \mathop {\hbox{max} }\limits_{{u_{k} }} \left\{ {g_{k} (x,u_{k} ;u_{k - } ) + W\left[ {k + 1,f_{k} (x,u_{k} ;u_{k - } );u_{(k + 1) - } } \right]} \right\}. $$
(A.3)
The maximization operator involves \( u_{k}^{{}} \). Again, the current state \( x \) and the previously executed controls \( u_{k - }^{{}} \) appear in the stage \( k \) optimization problem. If the first-order conditions of the maximization problem in (A.3) satisfy the implicit function theorem, one can obtain the optimal controls \( u_{k}^{{}} \) as functions of \( x \) and \( u_{k - }^{{}} \). Substituting these optimal controls into the function on the RHS of (A.3) yields the function \( W(k,x;u_{k - }^{{}} ) \). □
Appendix B: Proof of Theorem 3.3
Using (3.12), one can obtain
$$ \begin{aligned} \xi^{i} \left( {k + 1,x_{k + 1}^{*} ;\underline{u}_{k - }^{*} } \right) & = B_{k + 1}^{i} \left( {x_{k + 1}^{*} ;\underline{u}_{k - }^{*} } \right) \\ & \quad + \;\left\{ {\sum\limits_{\zeta = k + 2}^{T} {B_{\zeta }^{i} \left( {x_{\zeta }^{*} ;\underline{u}_{\zeta - }^{*} } \right) + q_{T + 1}^{i} \left( {x_{T + 1} ;\underline{u}_{(T + 1) - }^{*} } \right)} } \right\}, \\ \end{aligned} $$
(B.1)
Upon substituting (B.1) into (3.12) yields
$$ \xi_{{}}^{i} (k,x_{k}^{*} ;\underline{u}_{k - }^{*} ) = B_{k}^{i} (x_{k}^{*} ;\underline{u}_{k - }^{*} ) + \xi_{{}}^{i} (k + 1,x_{k + 1}^{*} ;\underline{u}_{(k + 1) - }^{*} ), $$
which can be expressed as
$$ \xi^{i} \left( {k,x_{k}^{*} ;\underline{u}_{k - }^{*} } \right) = B_{k}^{i} \left( {x_{k}^{*} ;\underline{u}_{k - }^{*} } \right) + \xi^{i} \left( {k + 1,f_{k} \left( {x_{k}^{*} ,\underline{u}_{k}^{*} ;\underline{u}_{k - }^{*} } \right);\underline{u}_{(k + 1) - }^{*} } \right). $$
(B.2)
From (B.2), one can obtain Theorem 3.3. □
Appendix C: Proof of Proposition 4.1
From the terminal payoff in (4.5), we have
$$ \begin{aligned} & W\left( {T + 1,x,y;\underline{u}_{(T + 1) - }^{(2)} ,\underline{u}_{(T + 1) - }^{(3)} ,\underline{u}_{(T + 1) - }^{(4)} } \right) \\ & \quad = \sum\limits_{i = 1}^{n} {\left( {Q_{T + 1}^{(x)i} x_{T + 1} + Q_{T + 1}^{(y)i} y_{T + 1} + \sum\limits_{\tau = T - 1}^{T} {p_{T + 1}^{(\tau )i} u_{\tau }^{(3)i} } + \varpi_{T + 1}^{i} } \right)\delta^{T} } , \\ \end{aligned} $$
and therefore, we obtain
$$ A_{T + 1} = \sum\limits_{i = 1}^{n} {Q_{T + 1}^{(x)i} } ,{\kern 1pt} \quad B_{T + 1} = \sum\limits_{i = 1}^{n} {Q_{T + 1}^{(y)i} } ,\;{\text{and}}\;C_{T + 1} = \sum\limits_{i = 1}^{n} {\sum\limits_{\tau = T - 1}^{T} {\left( {p_{T + 1}^{(\tau )i} u_{\tau }^{(3)i} + \varpi_{T + 1}^{i} } \right)} } . $$
(C.1)
We begin with the last operating stage and consider the problem in stage \( T \)
$$ \begin{aligned} W\left( {T,x,y;\underline{u}_{T - }^{(2)} ,\underline{u}_{T - }^{(3)} ,\underline{u}_{T - }^{(4)} } \right) & = \mathop {\hbox{max} }\limits_{\substack{ u_{T}^{(1)i} ,u_{T}^{(2)i} ,\,u_{T}^{(3)i} ,u_{T}^{(4)i} \\ i \in N\; } } \left\{ {\sum\limits_{i = 1}^{n} {\left( {\alpha_{T}^{i} x - c_{T}^{i} \left( {u_{T}^{(2)i} } \right)^{2} } \right.} } \right. \\ & \quad - \;\gamma_{T}^{i} \left( {u_{T}^{(3)i} } \right)^{2} + \sum\limits_{\tau = T - 2}^{T} {p_{T}^{(\tau )i} u_{\tau }^{(3)i} } + \left[ {P_{T}^{i} u_{T}^{(4)i} - w_{T}^{i} (u_{T}^{(4)i} )^{2} } \right] - \left. {\varepsilon_{T}^{i} \left( {u_{T}^{(1)i} } \right)^{2} - h_{T}^{i} y} \right)\delta^{T - 1} \\ & \left. {\quad + \;\sum\limits_{i = 1}^{n} {\left( {Q_{T + 1}^{(x)i} x_{T + 1} + Q_{T + 1}^{(y)i} y_{T + 1} + \sum\limits_{\tau = T - 1}^{T} {p_{T + 1}^{(\tau )i} u_{\tau }^{(3)i} + \varpi_{T + 1}^{i} } } \right)} \delta^{T} } \right\}, \\ \end{aligned} $$
(C.2)
where \( x_{T + 1} = x + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 1}^{T} {a_{T}^{(\tau )j} u_{\tau }^{(2)j} } - \lambda \,x} , \)\( y_{T + 1} = y + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 3}^{T} {m_{T}^{(\tau )j} u_{\tau }^{(4)j} } } - \sum\limits_{j = 1}^{n} {b^{j} u_{T}^{(1)j} } (y)^{1/2} - \vartheta \,y \).Performing the indicated maximization in (C.2) yields the optimal cooperative strategies:
$$ \begin{aligned} u_{T}^{(1)i} & = - B_{T + 1} \frac{{b^{i} (y)^{1/2} \delta }}{{2\varepsilon_{T}^{i} }}, \\ u_{T}^{(2)i} & = \delta A_{T + 1} \frac{{a_{T}^{(T)i} }}{{2c_{T}^{i} }}, \\ u_{T}^{(3)i} & = \left( {p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} } \right)\frac{1}{{2\gamma_{T}^{i} }},\;{\text{and}} \\ u_{T}^{(4)i} & = \frac{{P_{T}^{i} + \delta B_{T + 1}^{{}} m_{T}^{(T)i} }}{{2w_{T}^{i} }},i \in N. \\ \end{aligned} $$
(C.3)
Invoking Proposition 4.1 and substituting (C.3) into the stage \( T \) equation in (C.2), we obtain:
$$ \begin{aligned} W\left( {T,x,y;\underline{u}_{T - }^{(2)} ,\underline{u}_{T - }^{(3)} ,\underline{u}_{T - }^{(4)} } \right) & = \left( {A_{T} x + B_{T} y + C_{T} } \right)\delta^{T - 1} \\ & = \sum\limits_{i = 1}^{n} {\left[ {\alpha_{T}^{i} x - c_{T}^{i} \left( {\frac{{\delta A_{T + 1}^{{}} a_{T}^{(T)i} }}{{2c_{T}^{i} }}\,} \right)^{2} - \gamma_{T}^{i} \left( {\frac{{p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} }}{{2\gamma_{T}^{i} }}} \right)^{2} } \right.} \\ & \quad + \;p_{T}^{(T)i} \left( {\frac{{p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} }}{{2\gamma_{T}^{i} }}} \right) + \sum\limits_{\tau = T - 2}^{T - 1} {p_{T}^{(\tau )i} u_{\tau }^{(3)i} } + P_{T}^{i} \left( {\frac{{P_{T}^{i} + \delta A_{T + 1} m_{T}^{(T)i} }}{{2w_{T}^{i} }}} \right) \\ & \left. {\quad - \;w_{T}^{i} \left( {\frac{{P_{T}^{i} + \delta A_{T + 1} m_{T}^{(T)i} }}{{2w_{T}^{i} }}} \right)^{2} - \varepsilon_{T}^{i} \left( {B_{T + 1} \frac{{b^{i} (y)^{1/2} \delta }}{{2\varepsilon_{T}^{i} }}} \right)^{2} - h_{T}^{i} y} \right]\delta^{T - 1} \\ & \quad + \;\left[ {A_{T + 1} \left( {x + \sum\limits_{j = 1}^{n} {a_{T}^{(T)j} \left( {\frac{{\delta A_{T + 1} a_{T}^{(T)j} }}{{2c_{T}^{j} }}} \right)} + \sum\limits_{j = 1}^{n} {a_{T}^{(T - 1)j} u_{T - 1}^{(2)j} - \lambda \,x} } \right)} \right. \\ & \quad + \;B_{T + 1} \left( {y + \sum\limits_{j = 1}^{n} {m_{T}^{(T)j} \frac{{P_{T}^{j} + \delta A_{T + 1} m_{T}^{(T)j} }}{{2w_{T}^{j} }}} + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 3}^{T - 1} {m_{T}^{(\tau )j} u_{\tau }^{(4)j} } } + \sum\limits_{j = 1}^{n} {\delta B_{T + 1} \frac{{(b^{j} )^{2} y}}{{2\varepsilon_{T}^{j} }} - \vartheta \,y} } \right) \\ & \left. {\quad + \;\sum\limits_{i = 1}^{n} {p_{T + 1}^{(T)i} \frac{{p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} }}{{2\gamma_{T}^{i} }}} + \sum\limits_{i = 1}^{n} {p_{T + 1}^{(T - 1)i} u_{T - 1}^{(3)i} } + \sum\limits_{i = 1}^{n} {\varpi_{T + 1}^{i} } } \right]\delta^{T} . \\ \end{aligned} $$
(C.4)
Both the left-hand side and right-hand side of (C.4) are linear functions of \( x \) and \( y \). For (C.4) to hold, it is required that:
$$ \begin{aligned} A_{T} & = \sum\limits_{i = 1}^{n} {\alpha_{T}^{i} + A_{T + 1} (1 - \lambda )\delta ,} \\ B_{T} & = \sum\limits_{i = 1}^{n} {\left[ { - \varepsilon_{T}^{i} \left( {\frac{{\delta B_{T + 1} b^{i} }}{{2\varepsilon_{T}^{i} }}} \right)^{2} - h_{T}^{i} } \right]} + \left[ {B_{T + 1} \left( {1 + \sum\limits_{j = 1}^{n} {b_{{}}^{j} } \frac{{\delta B_{T + 1} b^{j} }}{{2\varepsilon_{T}^{j} }} - \vartheta \,} \right)} \right]\delta , \\ \end{aligned} $$
(C.5)
and \( C_{T}^{{}} \) is an expression involving previously executed controls \( \underline{u}_{T - }^{(2)} ,\underline{u}_{T - }^{(3)} ,\underline{u}_{T - }^{(4)} \) which can be expressed as:
$$ \begin{aligned} C_{T} & = \sum\limits_{i = 1}^{n} {\left[ { - c_{T}^{i} \left( {\frac{{\delta A_{T + 1} a_{T}^{(T)i} }}{{2c_{T}^{i} }}} \right)^{2} - \gamma_{T}^{i} \left( {\frac{{p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} }}{{2\gamma_{T}^{i} }}} \right)^{2} + p_{T}^{(T)i} \left( {\frac{{p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} }}{{2\gamma_{T}^{i} }}} \right)} \right.} \\ & \left. {\quad + \;\sum\limits_{\tau = T - 2}^{T - 1} {p_{T}^{(\tau )i} u_{\tau }^{(3)i} } + P_{T}^{i} \left( {\frac{{P_{T}^{i} + \delta A_{T + 1}^{{}} m_{T}^{(T)i} }}{{2w_{T}^{i} }}} \right) - w_{T}^{i} \left( {\frac{{P_{T}^{i} + \delta A_{T + 1}^{{}} m_{T}^{(T)i} }}{{2w_{T}^{i} }}} \right)^{2} } \right] \\ & \quad + \;\left[ {A_{T + 1} \left( {\sum\limits_{j = 1}^{n} {a_{T}^{(T)j} \left( {\frac{{\delta A_{T + 1} a_{T}^{(T)j} }}{{2c_{T}^{j} }}} \right)} + \sum\limits_{j = 1}^{n} {a_{T}^{(T - 1)j} u_{T - 1}^{(2)j} } } \right)} \right. \\ & \quad + \;B_{T + 1} \left( {\sum\limits_{j = 1}^{n} {m_{T}^{(T)j} \frac{{P_{T}^{j} + \delta A_{T + 1}^{{}} m_{T}^{(T)j} }}{{2w_{T}^{j} }}} + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 3}^{T - 1} {} m_{T}^{(\tau )j} u_{\tau }^{(4)j} } } \right) \\ & \left. {\quad + \;\sum\limits_{i = 1}^{n} {p_{T + 1}^{(T)i} \frac{{p_{T}^{(T)i} + \delta {\kern 1pt} p_{T + 1}^{(T)i} }}{{2\gamma_{T}^{i} }}} + \sum\limits_{i = 1}^{n} {p_{T + 1}^{(T - 1)i} u_{T - 1}^{(3)i} } + \sum\limits_{i = 1}^{n} {\varpi_{T + 1}^{i} } } \right]\delta \\ & = \;\varOmega_{T} + \sum\limits_{i = 1}^{n} {\sum\limits_{\tau = T - 2}^{T - 1} {p_{T}^{(\tau )i} u_{\tau }^{(3)i} } } + \delta A_{T + 1} \sum\limits_{j = 1}^{n} {a_{T}^{(T - 1)j} u_{T - 1}^{(2)j} } + \delta B_{T + 1} \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 3}^{T - 1} {m_{T}^{(\tau )j} u_{\tau }^{(4)j} } } \\ & \quad + \;\delta \sum\limits_{i = 1}^{n} {p_{T + 1}^{(T - 1)i} u_{T - 1}^{(3)i} } . \\ \end{aligned} $$
(C.6)
Then, we move to the problem in stage \( T - 1 \). Using \( W(T,x,y;\underline{u}_{T - }^{(2)} ,\underline{u}_{T - }^{(3)} ,\underline{u}_{T - }^{(4)} ) \) in (C.4), the \( T - 1 \) stage equation in (4.6) can be expressed as
$$ \begin{aligned} W\left( {T - 1,x,y;\underline{u}_{(T - 1) - }^{(2)} ,\underline{u}_{(T - 1) - }^{(3)} ,\underline{u}_{(T - 1) - }^{(4)} } \right) & = \mathop {\hbox{max} }\limits_{\substack{ u_{T - 1}^{(1)i} ,u_{T - 1}^{(2)i} ,\,u_{T - 1}^{(3)i} ,u_{T - 1}^{(4)i} \\ i \in N\; } } \left\{ {\sum\limits_{i = 1}^{n} {\left( {\alpha_{T - 1}^{i} x - c_{T - 1}^{i} \left( {u_{T - 1}^{(2)i} } \right)^{2} } \right.} } \right. \\ & \quad \left. { - \;\gamma_{T - 1}^{i} \left( {u_{T - 1}^{(3)i} } \right)^{2} + \sum\limits_{\tau = T - 3}^{T - 1} {p_{T - 1}^{(\tau )i} u_{\tau }^{(3)i} } + \left[ {P_{T - 1}^{i} u_{T - 1}^{(4)i} - w_{T - 1}^{i} (u_{T - 1}^{(4)i} )^{2} } \right] - \varepsilon_{T - 1}^{i} \left( {u_{T - 1}^{(1)i} } \right)^{2} - h_{T - 1}^{i} y} \right)\delta^{T - 2} \\ & \left. {\quad + \;\left( {A_{T} x + B_{T} y + C_{T} } \right)\delta^{T - 1} } \right\}, \\ \end{aligned} $$
(C.7)
where \( x_{T} = x + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 2}^{T - 1} {a_{T - 1}^{(\tau )j} u_{\tau }^{(2)j} } - \lambda \,x} , \)\( y_{T} = y + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 4}^{T - 1} {m_{T - 1}^{(\tau )j} u_{\tau }^{(4)j} } } - \sum\limits_{j = 1}^{n} {b^{j} u_{T - 1}^{(1)j} } (y)^{1/2} - \vartheta \,y. \)
Performing the indicated maximization in (C.7) yields the optimal cooperative strategies:
$$ \begin{aligned} u_{T - 1}^{(1)i} & = \frac{{ - B_{T}^{{}} b_{{}}^{i} (y)^{1/2} \delta }}{{2\varepsilon_{T - 1}^{i} }}, \\ u_{T - 1}^{(2)i} & = \frac{{\delta A_{T} a_{T - 1}^{(T - 1)i} + \delta^{2} A_{T + 1} a_{T}^{(T - 1)i} }}{{2c_{T - 1}^{i} }}, \\ u_{T - 1}^{(3)i} & = \frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }},\;{\text{and}} \\ u_{T - 1}^{(4)i} & = \frac{{P_{T - 1}^{i} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)i} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)i} }}{{2w_{T - 1}^{i} }},\;i \in N. \\ \end{aligned} $$
(C.8)
Invoking “Proposition 4.1” and substituting (C.8) into the stage \( T - 1 \) equation in (4.6), we obtain:
$$ \begin{aligned} W(T - 1,x,y;\underline{u}_{(T - 1) - }^{(2)} ,\underline{u}_{(T - 1) - }^{(3)} ,\underline{u}_{(T - 1) - }^{(4)} ) & = (A_{T - 1}^{{}} x + B_{T - 1}^{{}} y + C_{T - 1}^{{}} )\delta^{T - 2} \\ & = \;\sum\limits_{i = 1}^{n} {} \left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\alpha_{T - 1}^{i} x - c_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta A_{T}^{{}} a_{T - 1}^{(T - 1)i} + \delta^{2} A_{T + 1}^{{}} a_{T}^{(T - 1)i} }}{{2c_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} \\ & \quad - \;\gamma_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} + p_{T - 1}^{(T - 1)} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;\sum\limits_{\tau = T - 3}^{T - 2} {} p_{T - 1}^{(\tau )i} u_{\tau }^{(3)i} + P_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{P_{T - 1}^{i} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)i} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)i} }}{{2w_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad - \;w_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{P_{T - 1}^{i} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)i} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)i} }}{{2w_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} - \varepsilon_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{B_{T}^{{}} b_{{}}^{i} (y)^{1/2} \delta }}{{2\varepsilon_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} - h_{T - 1}^{i} y\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]\delta^{T - 2} \\ & \quad + \;\left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.A_{T}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.x + \sum\limits_{j = 1}^{n} {a_{T - 1}^{(T - 1)j} } \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta A_{T}^{{}} a_{T - 1}^{(T - 1)j} + \delta^{2} A_{T + 1}^{{}} a_{T}^{(T - 1)j} }}{{2c_{T - 1}^{j} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) + \sum\limits_{j = 1}^{n} {} a_{T - 1}^{(T - 2)j} u_{T - 2}^{(2)j} - \lambda \,x\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;B_{T}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.y + \sum\limits_{j = 1}^{n} {} m_{T - 1}^{(T - 1)j} \frac{{P_{T - 1}^{j} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)j} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)j} }}{{2w_{T - 1}^{j} }} \\ & \quad + \;\sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 4}^{T - 2} {} m_{T - 1}^{(\tau )j} u_{\tau }^{(4)j} } + \sum\limits_{j = 1}^{n} {} \delta B_{T}^{{}} \frac{{(b_{{}}^{j} )^{2} y}}{{2\varepsilon_{T - 1}^{j} }} - \vartheta \,y\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;\left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\varOmega_{T}^{{}} + \sum\limits_{i = 1}^{n} {} p_{T}^{(T - 1)i} \frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }} + \sum\limits_{i = 1}^{n} {} p_{T}^{(T - 2)i} u_{T - 2}^{(3)i} \\ & \quad + \;\delta A_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {} a_{T}^{(T - 1)j} \frac{{\delta A_{T}^{{}} a_{T - 1}^{(T - 1)j} + \delta^{2} A_{T + 1}^{{}} a_{T}^{(T - 1)j} }}{{2c_{T - 1}^{j} }} \\ & \quad + \;\delta B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {m_{T}^{(T - 1)j} } \frac{{P_{T - 1}^{j} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)j} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)j} }}{{2w_{T - 1}^{j} }} + \delta B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} \begin{aligned} \sum\limits_{\tau = T - 3}^{T - 2} {} m_{T}^{(\tau )j} u_{\tau }^{(4)j} \hfill \\ \hfill \\ \end{aligned} \\ & \quad + \;\delta \sum\limits_{i = 1}^{n} {} p_{T + 1}^{(T - 1)i} \frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]\delta^{T - 1} . \\ \end{aligned} $$
(C.9)
Both the left-hand side and right-hand side of (C.9) are linear functions of \( x \) and \( y \). For (C.9) to hold, it is required that:
$$ \begin{aligned} A_{T - 1}^{{}} & = \sum\limits_{i = 1}^{n} {} \alpha_{T - 1}^{i} + A_{T}^{{}} (1 - \lambda )\delta , \\ B_{T - 1}^{{}} & = \sum\limits_{i = 1}^{n} {} \left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right. - \varepsilon_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta B_{T}^{{}} b_{{}}^{i} }}{{2\varepsilon_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} - h_{T - 1}^{i} \left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.B_{T}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.1 + \sum\limits_{j = 1}^{n} {b_{{}}^{j} } \frac{{\delta B_{T}^{{}} b_{{}}^{j} }}{{2\varepsilon_{T - 1}^{j} }} - \vartheta \,\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]\delta , \\ \end{aligned} $$
(C.10)
and \( C_{T - 1}^{{}} \) is an expression involving previously executed controls \( \underline{u}_{(T - 1) - }^{(2)} ,\underline{u}_{(T - 1) - }^{(3)} ,\underline{u}_{(T - 1) - }^{(4)} \) which can be expressed as:
$$ \begin{aligned} C_{T - 1}^{{}} & = \;\sum\limits_{i = 1}^{n} {} \left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right. - c_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta A_{T}^{{}} a_{T - 1}^{(T - 1)i} + \delta^{2} A_{T + 1}^{{}} a_{T}^{(T - 1)i} }}{{2c_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} \\ & \quad - \;\gamma_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} + p_{T - 1}^{(T - 1)} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;\sum\limits_{\tau = T - 3}^{T - 2} {} p_{T - 1}^{(\tau )i} u_{\tau }^{(3)i} + P_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{P_{T - 1}^{i} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)i} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)i} }}{{2w_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad - \;w_{T - 1}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{P_{T - 1}^{i} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)i} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)i} }}{{2w_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} \left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right] \\ & \quad + \;\left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.A_{T}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\sum\limits_{j = 1}^{n} {a_{T - 1}^{(T - 1)j} } \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta A_{T}^{{}} a_{T - 1}^{(T - 1)j} + \delta^{2} A_{T + 1}^{{}} a_{T}^{(T - 1)j} }}{{2c_{T - 1}^{j} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) + \sum\limits_{j = 1}^{n} {} a_{T - 1}^{(T - 2)j} u_{T - 2}^{(2)j} \left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;B_{T}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\sum\limits_{j = 1}^{n} {} m_{T - 1}^{(T - 1)j} \frac{{P_{T - 1}^{j} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)j} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)j} }}{{2w_{T - 1}^{j} }} + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 4}^{T - 2} {} m_{T - 1}^{(\tau )j} u_{\tau }^{(4)j} \left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)} \\ & \quad + \;\left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\varOmega_{T}^{{}} + \sum\limits_{i = 1}^{n} {} p_{T}^{(T - 1)i} \frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }} + \sum\limits_{i = 1}^{n} {} p_{T}^{(T - 2)i} u_{T - 2}^{(3)i} \\ & \quad + \;\delta A_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {} a_{T}^{(T - 1)j} \frac{{\delta A_{T}^{{}} a_{T - 1}^{(T - 1)j} + \delta^{2} A_{T + 1}^{{}} a_{T}^{(T - 1)j} }}{{2c_{T - 1}^{j} }} \\ & \quad + \;\delta B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {m_{T}^{(T - 1)j} } \frac{{P_{T - 1}^{j} + \delta B_{T}^{{}} m_{T - 1}^{(T - 1)j} + \delta^{2} B_{T + 1}^{{}} m_{T}^{(T - 1)j} }}{{2w_{T - 1}^{j} }} + \delta B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 3}^{T - 2} {} m_{T}^{(\tau )j} u_{\tau }^{(4)j} } \\ & \quad + \;\delta \sum\limits_{i = 1}^{n} {} p_{T + 1}^{(T - 1)i} \frac{{p_{T - 1}^{(T - 1)i} + \delta {\kern 1pt} p_{T}^{(T - 1)i} + \delta^{2} {\kern 1pt} p_{T + 1}^{(T - 1)i} }}{{2\gamma_{T - 1}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]\delta \\ & = \;\varOmega_{T - 1}^{{}} + \sum\limits_{i = 1}^{n} {} \sum\limits_{\tau = T - 3}^{T - 2} {} p_{T - 1}^{(\tau )i} u_{\tau }^{(3)i} + \delta A_{T}^{{}} \sum\limits_{j = 1}^{n} {} a_{T - 1}^{(T - 2)j} u_{T - 2}^{(2)j} + \delta B_{T} \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 4}^{T - 2} {} m_{T - 1}^{(\tau )j} u_{\tau }^{(4)j} } \\ & \quad + \;\delta \sum\limits_{i = 1}^{n} {} p_{T}^{(T - 2)i} u_{T - 2}^{(3)i} + \delta^{2} B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 3}^{T - 2} {} m_{T}^{(\tau )j} u_{\tau }^{(4)j} } . \\ \end{aligned} $$
(C.11)
Then,
we move to the problem in stage \( T - 2 \). Using \( W(T - 1,x,y;\underline{u}_{(T - 1) - }^{(2)} ,\underline{u}_{(T - 1) - }^{(3)} ,\underline{u}_{(T - 1) - }^{(4)} ) \) in (C.9), the \( T - 2 \) stage equation in (4.6) can be expressed as
$$ \begin{aligned} W(T - 2,x,y;\underline{u}_{(T - 2) - }^{(2)} ,\underline{u}_{(T - 2) - }^{(3)} ,\underline{u}_{(T - 2) - }^{(4)} ) & = \mathop {\hbox{max} }\limits_{\substack{ u_{T - 2}^{(1)i} ,u_{T - 2}^{(2)i} ,\,u_{T - 2}^{(3)i} ,u_{T - 2}^{(4)i} \\ i \in N\; } } \left\{ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\sum\limits_{i = 1}^{n} {\left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.} \alpha_{T - 2}^{i} x - c_{T - 2}^{i} (u_{T - 2}^{(2)i} )^{2} \\ & \quad - \;\gamma_{T - 2}^{i} (u_{T - 2}^{(3)i} )^{2} + \sum\limits_{\tau = T - 4}^{T - 2} {} p_{T - 2}^{(\tau )i} u_{\tau }^{(3)i} + [P_{T - 2}^{i} u_{T - 2}^{(4)i} - w_{T - 2}^{i} (u_{T - 2}^{(4)i} )^{2} ] - \varepsilon_{T - 2}^{i} (u_{T - 2}^{(1)i} )^{2} - h_{T - 2}^{i} y\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)\delta^{T - 2} \\ & \quad + \;(A_{T - 1}^{{}} x + B_{T - 1}^{{}} y + C_{T - 1}^{{}} )\delta^{T - 2} \left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right\}, \\ \end{aligned} $$
(C.12)
where \( x_{T - 1}^{{}} = x + \sum\limits_{j = 1}^{n} {} \sum\limits_{\tau = T - 3}^{T - 2} {a_{T - 2}^{(\tau )j} u_{\tau }^{(2)j} } - \lambda \,x \),\( y_{T - 1}^{{}} = y + \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 5}^{T - 2} {} m_{T - 2}^{(\tau )j} u_{\tau }^{(4)j} } - \sum\limits_{j = 1}^{n} {b_{{}}^{j} u_{T - 2}^{(1)j} } (y)^{1/2} - \vartheta \,y \).
Performing the indicated maximization in (C.12) yields the optimal cooperative strategies:
$$ \begin{aligned} u_{T - 2}^{(1)i} & = \frac{{ - B_{T - 1}^{{}} b_{{}}^{i} (y)^{1/2} \delta }}{{2\varepsilon_{T - 2}^{i} }}, \\ u_{T - 2}^{(2)i} & = \frac{{\delta A_{T - 1}^{{}} a_{T - 2}^{(T - 2)i} + \delta^{2} A_{T}^{{}} a_{T - 1}^{(T - 2)i} }}{{2c_{T - 2}^{i} }}, \\ u_{T - 2}^{(3)i} & = \frac{{p_{T - 2}^{(T - 2)i} + \delta {\kern 1pt} p_{T - 1}^{(T - 2)i} + \delta^{2} {\kern 1pt} p_{T}^{(T - 2)i} }}{{2\gamma_{T - 2}^{i} }},\;{\text{and}} \\ u_{T - 2}^{(4)i} & = \frac{{P_{T - 2}^{i} + \delta B_{T - 1}^{{}} m_{T - 2}^{(T - 2)i} + \delta^{2} B_{T}^{{}} m_{T - 1}^{(T - 2)i} + \delta^{3} B_{T + 1}^{{}} m_{T}^{(T - 2)i} }}{{2w_{T - 2}^{i} }},i \in N. \\ \end{aligned} $$
(C.13)
Invoking Proposition 4.1 and substituting (C.13) into the stage \( T - 2 \) equation in (4.6), we obtain:
$$ \begin{aligned} W(T - 2,x,y;\underline{u}_{(T - 2) - }^{(2)} ,\underline{u}_{(T - 2) - }^{(3)} ,\underline{u}_{(T - 2) - }^{(4)} ) & = (A_{T - 2}^{{}} x + B_{T - 2}^{{}} y + C_{T - 2}^{{}} )\delta^{T - 3} \\ & = \;\sum\limits_{i = 1}^{n} {} \left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\alpha_{T - 2}^{i} x - c_{T - 2}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta A_{T - 1}^{{}} a_{T - 2}^{(T - 2)i} + \delta^{2} A_{T}^{{}} a_{T - 1}^{(T - 2)i} }}{{2c_{T - 2}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} \\ & \quad - \;\gamma_{T - 2}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{p_{T - 2}^{(T - 2)i} + \delta {\kern 1pt} p_{T - 1}^{(T - 2)i} + \delta^{2} {\kern 1pt} p_{T}^{(T - 2)i} }}{{2\gamma_{T - 2}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} + p_{T - 2}^{(T - 2)} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{p_{T - 2}^{(T - 2)i} + \delta {\kern 1pt} p_{T - 1}^{(T - 2)i} + \delta^{2} {\kern 1pt} p_{T}^{(T - 2)i} }}{{2\gamma_{T - 2}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;\sum\limits_{\tau = T - 4}^{T - 3} {} p_{T - 2}^{(\tau )i} u_{\tau }^{(3)i} + P_{T - 2}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{P_{T - 2}^{i} + \delta B_{T - 1}^{{}} m_{T - 2}^{(T - 2)i} + \delta^{2} B_{T}^{{}} m_{T - 1}^{(T - 2)i} + \delta^{3} B_{T + 1}^{{}} m_{T}^{(T - 2)i} }}{{2w_{T - 2}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad - \;w_{T - 2}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{P_{T - 2}^{i} + \delta B_{T - 1}^{{}} m_{T - 2}^{(T - 2)i} + \delta^{2} B_{T}^{{}} m_{T - 1}^{(T - 2)i} + \delta^{3} B_{T + 1}^{{}} m_{T}^{(T - 2)i} }}{{2w_{T - 2}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} \\ & \quad - \;\varepsilon_{T - 2}^{i} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{B_{T - 1}^{{}} b_{{}}^{i} (y)^{1/2} \delta }}{{2\varepsilon_{T - 2}^{i} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)^{2} - h_{T - 2}^{i} y\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]\delta^{T - 3} \\ & \quad + \;\left[ {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.A_{T - 1}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.x + \sum\limits_{j = 1}^{n} {a_{T - 2}^{(T - 2)j} } \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\frac{{\delta A_{T - 1}^{{}} a_{T - 2}^{(T - 2)j} + \delta^{2} A_{T}^{{}} a_{T - 1}^{(T - 2)j} }}{{2c_{T - 2}^{j} }}\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) + \sum\limits_{j = 1}^{n} {} a_{T - 2}^{(T - 3)j} u_{T - 3}^{(2)j} - \lambda \,x\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;B_{T - 1}^{{}} \left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.y + \sum\limits_{j = 1}^{n} {} m_{T - 2}^{(T - 2)j} \frac{{P_{T - 2}^{j} + \delta B_{T - 1}^{{}} m_{T - 2}^{(T - 2)j} + \delta^{2} B_{T}^{{}} m_{T - 1}^{(T - 2)j} + \delta^{3} B_{T + 1}^{{}} m_{T}^{(T - 2)j} }}{{2w_{T - 2}^{j} }} \\ & \quad + \;\sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 5}^{T - 3} {} m_{T - 2}^{(\tau )j} u_{\tau }^{(4)j} } + \sum\limits_{j = 1}^{n} {} \delta B_{T - 1}^{{}} \frac{{(b_{{}}^{j} )^{2} y}}{{2\varepsilon_{T - 2}^{j} }} - \vartheta \,y\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right) \\ & \quad + \;\left( {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right.\varOmega_{T - 1}^{{}} + \sum\limits_{i = 1}^{n} {} p_{T - 1}^{(T - 2)i} \frac{{p_{T - 2}^{(T - 2)i} + \delta {\kern 1pt} p_{T - 1}^{(T - 2)i} + \delta^{2} {\kern 1pt} p_{T}^{(T - 2)i} }}{{2\gamma_{T - 2}^{i} }} + \sum\limits_{i = 1}^{n} {} p_{T - 1}^{(T - 3)i} u_{T - 3}^{(3)i} \\ & \quad + \;\delta A_{T}^{{}} \sum\limits_{j = 1}^{n} {} a_{T - 1}^{(T - 2)j} \frac{{\delta A_{T - 1}^{{}} a_{T - 2}^{(T - 2)j} + \delta^{2} A_{T}^{{}} a_{T - 1}^{(T - 2)j} }}{{2c_{T - 2}^{j} }} \\ & \quad + \;\delta B_{T} \sum\limits_{j = 1}^{n} {m_{T - 1}^{(T - 2)j} } \frac{{P_{T - 2}^{i} + \delta B_{T - 1}^{{}} m_{T - 2}^{(T - 2)i} + \delta^{2} B_{T}^{{}} m_{T - 1}^{(T - 2)i} + \delta^{3} B_{T + 1}^{{}} m_{T}^{(T - 2)i} }}{{2w_{T - 2}^{i} }} \\ & \quad + \;\delta B_{T} \sum\limits_{j = 1}^{n} {\sum\limits_{\tau = T - 4}^{T - 3} {} m_{T - 1}^{(\tau )j} u_{\tau }^{(4)j} } + \delta \sum\limits_{i = 1}^{n} {} p_{T}^{(T - 2)i} \frac{{p_{T - 2}^{(T - 2)i} + \delta {\kern 1pt} p_{T - 1}^{(T - 2)i} + \delta^{2} {\kern 1pt} p_{T}^{(T - 2)i} }}{{2\gamma_{T - 2}^{i} }} \\ & \quad + \;\delta^{2} B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {m_{T}^{(T - 2)j} } \frac{{P_{T - 2}^{i} + \delta B_{T - 1}^{{}} m_{T - 2}^{(T - 2)i} + \delta^{2} B_{T}^{{}} m_{T - 1}^{(T - 2)i} + \delta^{3} B_{T + 1}^{{}} m_{T}^{(T - 2)i} }}{{2w_{T - 2}^{i} }} \\ & \quad + \;\delta^{2} B_{T + 1}^{{}} \sum\limits_{j = 1}^{n} {m_{T}^{(T - 3)j} u_{T - 3}^{(4)j} } \left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right)\left. {\begin{array}{*{20}c} {} \\ {} \\ \end{array} } \right]\delta_{{}}^{T - 3} . \\ \end{aligned} $$
(C.14)
Both the left-hand side and right-hand side of (C.14) are linear functions of \( x \) and \( y \). For (C.14) to hold, it is required that:
$$ \begin{aligned} A_{T - 2} & = \sum\limits_{i = 1}^{n} {\alpha_{T - 2}^{i} + A_{T - 1} (1 - \lambda )\delta } , \\ B_{T - 2} & = \sum\limits_{i = 1}^{n} {\left[ { - \varepsilon_{T - 2}^{i} \left( {\frac{{\delta B_{T - 1} b_{{}}^{i} }}{{2\varepsilon_{T - 2}^{i} }}} \right)^{2} - h_{T - 2}^{i} } \right]} + \left[ {B_{T - 1} \left( {1 + \sum\limits_{j = 1}^{n} {b^{j} } \frac{{\delta B_{T - 1} b^{j} }}{{2\varepsilon_{T - 2}^{j} }} - \vartheta } \right)} \right]\delta , \\ \end{aligned} $$
(C.15)
and \( C_{T - 2}^{{}} \) is an expression involving previously executed controls \( \underline{u}_{(T - 2) - }^{(2)} ,\underline{u}_{(T - 2) - }^{(3)} ,\underline{u}_{(T - 2) - }^{(4)} \).
We move to the problem in stage \( k \in \{ T - 3,T - 4, \cdots ,1\} \). Following the above analysis, we obtain
$$ \begin{aligned} A_{k} & = \sum\limits_{i = 1}^{n} {\alpha_{k}^{i} + A_{k + 1} (1 - \lambda )\delta } , \\ B_{k} & = \sum\limits_{i = 1}^{n} {\left[ { - \varepsilon_{k}^{i} \left( {\frac{{\delta B_{k + 1} b_{{}}^{i} }}{{2\varepsilon_{k}^{i} }}} \right)^{2} - h_{k}^{i} } \right]} + \left[ {B_{k} \left( {1 + \sum\limits_{j = 1}^{n} {b^{j} } \frac{{\delta B_{k + 1} b^{j} }}{{2\varepsilon_{k}^{j} }} - \vartheta \,} \right)} \right]\delta , \\ \end{aligned} $$
(C.16)
and \( C_{k}^{{}} \) is an expression involving previously executed controls \( \underline{u}_{k - }^{(2)} ,\underline{u}_{k - }^{(3)} ,\underline{u}_{k - }^{(4)} \).
The optimal cooperative strategies can be obtained as:
$$ \begin{aligned} u_{k}^{(1)i} & = \frac{{ - B_{k + 1} b^{i} (y)^{1/2} \delta }}{{2\varepsilon_{k}^{i} }}, \\ u_{k}^{(2)i} & = \frac{{\delta A_{k + 1} a_{k}^{(k)i} + \delta^{2} A_{k + 2} a_{k + 1}^{(k)i} }}{{2c_{k}^{i} }}, \\ u_{k}^{(3)i} & = \frac{{p_{k}^{(k)i} + \delta {\kern 1pt} p_{k + 1}^{(k)i} + \delta^{2} {\kern 1pt} p_{k + 2}^{(k)i} }}{{2\gamma_{k}^{i} }},{\text{and}} \\ u_{k}^{(4)i} & = \frac{{P_{k}^{i} + \delta B_{k + 1} m_{k}^{(k)i} + \delta^{2} B_{k + 2} m_{k + 1}^{(k)i} + \delta^{3} B_{k + 3} m_{k + 2}^{(k)i} + \delta^{4} B_{k + 4} m_{k + 3}^{(k)i} }}{{2w_{T - 2}^{i} }}, \\ \end{aligned} $$
(C.17)
for \( i \in N \) and \( k \in \{ T - 3,T - 4, \cdots ,1\} \).
For notational convenience, we specify \( A_{T + 2}^{{}} = 0 \), \( {\kern 1pt} p_{T + 2}^{(T)i} = 0 \) and \( B_{T + 2}^{{}} = B_{T + 3}^{{}} = B_{T + 4}^{{}} = 0 \). Then, the optimal cooperative strategies, for \( k \in \{ 1,2, \cdots ,T\} \), can be expressed as in (C.17).□
