Distributed Neuro-Dynamic Algorithm for Price-Based Game in Energy Consumption System

Abstract

In this paper, a plug-in hybrid electric vehicles energy consumption system is studied. In order to protect each player’s privacy, the information exchange is going on the neighboring players, and a connected undirected graph is used to pattern the information flow between the players. Hence, it is impossible for each player to access the aggregate electricity consumption directly, which determines the electricity price. Under the noncooperative game frame, a distributed neuro-dynamic algorithm is proposed to optimize the benefit of each individual player base on the pricing strategies. A dynamic average consensus is applied to estimate the aggregate consumption and a projection neural network is employed to seek the Nash equilibrium point. The convergence of the proposed distributed algorithm is analyzed through the Lyapunov stability analysis. Finally, the effectiveness of the distributed neuro-dynamic algorithm is manifested in the simulation.

Appendix

1.1 A. Proof of Lemma 1

Proof

From [24], we recall that $C_i$ is continuous and quasiconvex with respect to $\varOmega _i$, there exists an insular and stable Nash equilibrium $x^*=\left( x_{1}^{*},x_{2}^{*},\ldots ,x_{n}^{*} \right) ^T$, for all $\,\,i\in {\mathbb {N}}$, such that there are

$$\begin{aligned} \frac{\partial C_i}{\partial x_i}\left( x^* \right) =0,\ \frac{\partial ^2C_i}{\partial x_{i}^{2}}\left( x^* \right) >0. \end{aligned}$$

(16)

and the matrix

$$\begin{aligned} B=\left[ \begin{array}{cccc} \frac{\partial ^2C_1}{\partial x_{1}^{2}}\left( x^* \right) &{} \frac{\partial ^2C_1}{\partial x_1\partial x_2}\left( x^* \right) &{} \cdots &{} \frac{\partial ^2C_1}{\partial x_1\partial x_N}\left( x^* \right) \\ \frac{\partial ^2C_2}{\partial x_1\partial x_2}\left( x^* \right) &{} \frac{\partial ^2C_2}{\partial x_{2}^{2}}\left( x^* \right) &{} \cdots &{} \frac{\partial ^2C_2}{\partial x_2\partial x_N}\left( x^* \right) \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ \frac{\partial ^2C_N}{\partial x_1\partial x_N}\left( x^* \right) &{} \frac{\partial ^2C_N}{\partial x_2\partial x_N}\left( x^* \right) &{} \cdots &{} \frac{\partial ^2C_N}{\partial x_{N}^{2}}\left( x^* \right) \\ \end{array} \right] \end{aligned}$$

is strictly diagonally dominant. In this paper, the strategy set $\varOmega _i=\left\{ x_i|x_{i}^{\min }\le x_i\le x_{i}^{\max },\right. \left. \forall i\in {\mathbb {N}} \right\}$ satisfies the conditions in the Lemma 1, and the matrix B is strictly diagonally dominant,and hence, nonsingular, i.e., $a<\underset{i\in {\mathbb {N}}}{\min }\frac{hy_i}{\left( N-3 \right) \left( x_{i}^{\max } \right) ^2}$ for $\ N>3$, B is positive definite by the Gershgorin circle theorem [37, 38], such that $C_i:\varOmega \rightarrow R$ is continuous and $\forall x_{-i}\in \varOmega _{-i},z_i\rightarrow C_i\left( z_i,x_{-i} \right)$ is strictly convex with respect to $\varOmega _i$ for problem 1, the inequation (16) is also satisfied at the Nash equilibrium. Therefore, the uniqueness Nash equilibrium $x^*$ exists.

1.2 B. Proof of Lemma 2

Proof

Let $G=\left[ {\hat{G}},\alpha \right] \in R^{n\times n}$ be an orthonormal matrix and $\alpha ^TL=0$, $\alpha \in R^n$. Additionally, $v=G\left[ \begin{array}{c} {\hat{v}}\\ v_{\alpha }\\ \end{array} \right]$, ${\hat{v}}\in R^{n-1}$. (10) can be written as

$$\begin{aligned}&\left[ \begin{array}{c} {{\dot{X}}}\\ {\dot{{\hat{v}}}}\\ {\dot{v_{\alpha }}}\\ \end{array} \right] =\left[ \begin{matrix} -I-L&{} -L{\hat{G}}&{} 0_{n\times 1}\\ {\hat{G}}^TL&{} 0_{n-1\times n-1}&{} 0_{n-1\times 1}\\ 0_{1\times n}&{} 0_{1\times n-1}&{} 0\\ \end{matrix} \right] \left[ \begin{array}{c} X\\ {\hat{v}}\\ v_{\alpha }\\ \end{array} \right] +\left[ \begin{array}{c} Nx\\ 0_{n-1\times 1}\\ 0\\ \end{array} \right] \end{aligned}$$

(17a)

$$\begin{aligned}&{{\dot{x}}}=-\sigma {\hat{l}}\left( \left[ x_i-P_{\varOmega _i}\left( x_i-\left( \frac{\partial C_{G,i}}{\partial x_i}+p\left( X_i \right) +x_i\frac{\partial p\left( X_i \right) }{\partial X_i} \right) \right) \right] _{vec} \right) \end{aligned}$$

(17b)

the Eq. (17a) indicate $v_{\alpha }\left( t \right) =v\left( 0 \right)$.

Supposing the quasisteady states of X, ${\hat{v}}$ are $X^q$ and ${\hat{v}}^q$ respectively, therefore, there is

$$\begin{aligned} \left[ \begin{array}{c} {{\dot{X}}^q}\\ {\dot{{\hat{v}}}^q}\\ \end{array} \right] =0 \end{aligned}$$

(18)

for fixed x. Meanwhile, the $\left[ \begin{matrix} -I-L&{} -L{\hat{G}}\\ {\hat{G}}^TL&{} 0_{n-1\times n-1}\\ \end{matrix} \right]$ is Hurwitz [39], $X^q$ and ${\hat{v}}^q$ will be unique values. We can also obtain that

$$\begin{aligned} \left[ \begin{array}{c} {\dot{X^q}}\\ {{\dot{v}}^q}\\ \end{array} \right] =0 \end{aligned}$$

(19)

which reveals $X^q$ is the equilibrium of (10a). Based on the dynamic average consensus in Sect. 2.4, it is distinct to receive $X^q=1_N\sum _{i=1}^N{x_i}$ for the fixed x.

Define $\tau =\sigma t$ such that (17) can be written as (20) in the $\tau$ time scale.

$$\begin{aligned}&\left[ \begin{array}{c} \sigma \frac{dX}{d\tau }\\ \sigma \frac{d{\hat{v}}}{d\tau }\\ \end{array} \right] =\left[ \begin{matrix} -I-L&{} -L{\hat{G}}\\ {\hat{G}}^TL&{} 0_{n-1\times n-1}\\ \end{matrix} \right] \left[ \begin{array}{c} X\\ {\hat{v}}\\ \end{array} \right] +\left[ \begin{array}{c} Nx\\ 0_{n-1\times 1}\\ \end{array} \right] \end{aligned}$$

(20a)

$$\begin{aligned}&{{\dot{x}}}=-\sigma {\hat{l}}\left( \left[ x_i-P_{\varOmega _i}\left( x_i-\left( \frac{\partial C_{G,i}}{\partial x_i}+p\left( X_i \right) +x_i\frac{\partial p\left( X_i \right) }{\partial X_i} \right) \right) \right] _{vec} \right) \end{aligned}$$

(20b)

Let $\sigma =0$, X and ${\hat{v}}$ will stay at the quasisteady state which indicates $X=\left( \sum _{i=1}^N{x_i} \right) 1_N$ and (20b) will be transformed into (21).

$$\begin{aligned} \begin{aligned} \frac{dx}{d\tau }=&-{\hat{l}}\left( \left[ x_i-P_{\varOmega _i}\left( x_i-\left( \frac{\partial C_{G,i}}{\partial x_i}+p\left( X_i \right) +x_i\frac{\partial p\left( X_i \right) }{\partial X_i} \right) \right) \right] _{vec} \right) \\&=-\,{\hat{l}}\left( \left[ x_i-P_{\varOmega _i}\left( x_i-\frac{\partial C_i\left( x_i,x_{-i} \right) }{\partial x_i} \right) \right] _{vec} \right) \end{aligned} \end{aligned}$$

(21)

According to Lemma 1, there exists the Nash equilibrium point $x^*$ and $\frac{\partial C_i}{\partial x_i}\left( x^* \right) =0$ such that $P_{\varOmega _i}\left( x^*_i-\frac{\partial C_i\left( x^* \right) }{\partial x_i} \right) =x_{i}^{*}$, then

$$\begin{aligned}&\left[ \begin{matrix} -I-L&{} -L\\ L&{} 0_n\\ \end{matrix} \right] \left[ \begin{array}{c} X\left( x^* \right) \\ v^*\\ \end{array} \right] +\left[ \begin{array}{c} Nx^*\\ 0_n\\ \end{array} \right] =0 \end{aligned}$$

(22a)

$$\begin{aligned}&\left[ x_{i}^{*}-P_{\varOmega _i}\left( x_{i}^{*}-\frac{\partial C_i\left( x_{i}^{*},x_{-i}^{*} \right) }{\partial x_i} \right) \right] _{vec}=0 \end{aligned}$$

(22b)

$x^*$ is the equilibrium point of Problem 2. In [20], $X\left( x \right)$, ${\hat{v}}\left( x \right)$ are bounded since $\left[ \begin{matrix} -I-L&{} -L{\hat{G}}\\ {\hat{G}}^TL&{} 0_{n-1\times n-1}\\ \end{matrix} \right]$ is nonsingular, hence, $x\left( t \right)$ converges to the Nash equilibrium for the Problem 2. $\square$

1.3 C. Proof of Theorem 1

Proof

Let $H:\ \varOmega \subset R^n\rightarrow R^n$ be differentiable with respect to $\varOmega$, and defining a function $s:\ R^n\rightarrow R$ as fowllows

$$\begin{aligned} s\left( x \right) =\int _0^1{\left( x-x_0 \right) }^TH\left( x_0+t\left( x-x_0 \right) \right) dt \end{aligned}$$

(23)

where $\nabla H$ is symmetric with respect to $\varOmega$ and $\nabla s\left( x \right) ^T=H\left( x \right)$ [26].

For all $g\in R^m$ and all $u\in \varOmega$ [40], there is

$$\begin{aligned} \left( g-P_{\varOmega }\left( g \right) \right) ^T\left( P_{\varOmega }\left( g \right) -u \right) \ge 0 \end{aligned}$$

(24)

Let $g=x-H\left( x \right)$ and $u=x$, then

$$\begin{aligned} \left( x-H\left( x \right) -P_{\varOmega }\left( x-H\left( x \right) \right) \right) ^T\left( P_{\varOmega }\left( x-H\left( x \right) \right) -x \right) \ge 0 \end{aligned}$$

(25)

i.e.,

$$\begin{aligned} H\left( x \right) ^T\left\{ P_{\varOmega }\left( x-H\left( x \right) \right) -x \right\} \le -||\left( P_{\varOmega }\left( x-H\left( x \right) \right) -x \right) ||^2 \end{aligned}$$

(26)

Define the Lyapunov function below

$$\begin{aligned} V\left( x \right) =\int _0^1{\left( x-x^* \right) }^TH\left( x^*+\rho \left( x-x^* \right) \right) d\rho \end{aligned}$$

(27)

let $H\left( x \right) =\nabla {\mathcal {F}}\left( x \right)$, then $x^*\in \varOmega ^*$ since $\nabla H$ is symmetric and positive define, with the definition of potential game, then

$$\begin{aligned} \frac{dV\left( x \right) }{dt}= & {} \left( \frac{dV}{dx} \right) ^T\frac{dx}{dt} \nonumber \\= & {} H\left( x \right) ^T\left\{ -{\hat{l}}\left( \left[ x_i-P_{\varOmega _i}\left( x_i-\frac{\partial C_i\left( x_i,x_{-i} \right) }{\partial x_i} \right) \right] _{vec} \right) \right\} \nonumber \\= & {} -\,{\hat{l}}H\left( x \right) ^T\left( x-P_{\varOmega }\left( x-H\left( x \right) \right) \right) \nonumber \\\le & {} -\,{\hat{l}}||\left( P_{\varOmega }\left( x-H\left( x \right) \right) -x \right) ||^2\le 0 \end{aligned}$$

(28)

when the equality holds only if $P_{\varOmega }\left( x-H\left( x \right) \right) -x=0$, i.e., $\frac{dx}{dt}=0$, meanwhile, B is strictly diagonally dominant such that the potential function ${\mathcal {F}}\left( x \right)$ is strictly convex in x. According the diagonally strict convexity, a uniqueness Nash equilibrium point $x^*$ can also be obtained [41], then, the system is globally asymptotically stable. $\square$