Nash Equilibrium Seeking in Quadratic Noncooperative Games Under Two Delayed Information-Sharing Schemes

Abstract

In this paper, we propose non-model-based strategies for locally stable convergence to Nash equilibrium in quadratic noncooperative games where acquisition of information (of two different types) incurs delays. Two sets of results are introduced: (a) one, which we call cooperative scenario, where each player employs the knowledge of the functional form of his payoff and knowledge of other players’ actions, but with delays; and (b) the second one, which we term the noncooperative scenario, where the players have access only to their own payoff values, again with delay. Both approaches are based on the extremum seeking perspective, which has previously been reported for real-time optimization problems by exploring sinusoidal excitation signals to estimate the Gradient (first derivative) and Hessian (second derivative) of unknown quadratic functions. In order to compensate distinct delays in the inputs of the players, we have employed predictor feedback. We apply a small-gain analysis as well as averaging theory in infinite dimensions, due to the infinite-dimensional state of the time delays, in order to obtain local convergence results for the unknown quadratic payoffs to a small neighborhood of the Nash equilibrium. We quantify the size of these residual sets and corroborate the theoretical results numerically on an example of a two-player game with delays.

Appendix: Averaging and Small-Gain Theorems

Theorem A.1

(Averaging Theorem for FDEs [32]) Consider the delay system

$$\begin{aligned} {\dot{x}}(t)= & {} f(t/\epsilon ,x_t), \quad \forall t \ge 0, \end{aligned}$$

(134)

where $\epsilon$ is a real parameter, $x_t(\varTheta ) = x (t+\varTheta )$ for $-r\le \varTheta \le 0$, and $f : {\mathbb {R}}_{+} \times \varOmega \rightarrow {\mathbb {R}}^n$ is a continuous functional from a neighborhood $\varOmega$ of 0 of the supremum-normed Banach space $X = C([-r, 0]; {\mathbb {R}}^n)$ of continuous functions from $[-r, 0]$ to ${\mathbb {R}}^n$. Assume that $f(t,\varphi )$ is periodic in t uniformly with respect to $\varphi$ in compact subsets of $\varOmega$ and that f has a continuous Fréchet derivative $\partial f (t,\varphi )/\partial \varphi$ in $\varphi$ on ${\mathbb {R}}_{+} \times \varOmega$. If $y = y_0\in \varOmega$ is an exponentially stable equilibrium for the average system

$$\begin{aligned} {\dot{y}}(t)= & {} f_0(y_t), \quad \forall t\ge 0, \end{aligned}$$

(135)

where $f_0(\varphi )=\lim _{T\rightarrow \infty }\frac{1}{T} \int _{0}^{T} f(s,\varphi ) \hbox {d}s$, then, for some $\epsilon _0 > 0$ and $0 <\epsilon \le \epsilon _0$, there is a unique periodic solution $t \mapsto x^*(t,\epsilon )$ of (134) with the properties of being continuous in t and $\epsilon$, satisfying $|x^*(t, \epsilon ) - y_0| \le {\mathcal {O}}(\epsilon )$, for $t \in {\mathbb {R}}_{+}$, and such that there is $\rho >0$ so that if $x(\cdot ;\varphi )$ is a solution of (134) with $x(s) = \varphi$ and $|\varphi - y_0| < \rho$, then $|x(t)-x^*(t,\epsilon )| \le C e^{-\gamma (t-s)}$, for $C>0$ and $\gamma >0$.

Theorem A.2

(Small-Gain Theorem for ODE and Hyperbolic PDE Loops [31]) Consider generalized solutions of the following initial-boundary value problem

$$\begin{aligned}&{\dot{x}}(t)=F(x(t),u(z,t),v(t)), \quad \forall t \ge 0, \end{aligned}$$

(136)

$$\begin{aligned}&u_t(z,t)+cu_z(z,t)=a(z)u(z,t)+g(z,x(t),u(z,t))+f(z,t), \quad \forall (z,t) \in [0,1] \times {\mathbb {R}}_{+}, \end{aligned}$$

(137)

$$\begin{aligned}&u(0,t)=\varphi (d(t), u(z,t),x(t)), \quad \forall t \ge 0, \quad u(z,0)=u_0, \quad x(0)=x_0. \end{aligned}$$

(138)

The state of the system (136)–(138) is $(u(z,t),x(t))\in C^{0}([0,1]\times {\mathbb {R}}_{+}) \times {\mathbb {R}}^n$, while the other variables $d\in C^{0}({\mathbb {R}}_+;{\mathbb {R}}^q)$, $f\in C^{0}([0,1] \times {\mathbb {R}}_+)$ and $v\in C^{0}({\mathbb {R}}_+\,;{\mathbb {R}}^m)$ are external inputs. We assume that $(0,0) \in C^{0}([0,1])\times {\mathbb {R}}^n$ is an equilibrium point for the input-free system, i.e., $F(0,0,0)=0$, $g(z,0,0)=0$, and $\varphi (0,0,0)=0$. Now, we assume that the ODE subsystem satisfies the ISS property:

• (H1) There exist constants $M, \sigma >0$, $b_3, \gamma _3\ge 0$, such that for every $x_0\in {\mathbb {R}}^n$, $u\in C^{0}([0,1] \times {\mathbb {R}}_{+})$ and $v\in C^0({\mathbb {R}}_{+}\,;{\mathbb {R}}^{m})$ the unique solution $x \in C^{1}({\mathbb {R}}_{+}\,;{\mathbb {R}}^{n})$ of (136) with $x(0)=x_0$ satisfies the following estimate

  $$\begin{aligned} |x(t)| \le M |x_0| \exp (-\sigma t) + \max _{0\le s \le t}(\gamma _3 \Vert u(s)\Vert _{\infty }+b_3|v(s)|), \quad \forall t \ge 0. \end{aligned}$$

  (139)

  We next need to estimate the static gain of the interconnections. To this purpose, we employ the following further assumption.

• (H2) There exist constants $b_2,\gamma _1,\gamma _2,A,B \ge 0$ such that the following growth conditions hold for every $x\in C^{1}({\mathbb {R}}_{+};{\mathbb {R}}^{n})$, $u \in C^{0}([0,1]\times {\mathbb {R}}_{+})$ and $d \in C^{0}({\mathbb {R}}_{+};{\mathbb {R}}^{q})$:

  $$\begin{aligned} |g(z,x,u)|\le & {} A \Vert u\Vert _{\infty } + \gamma _1 |x|, \quad \forall z \in [0,1], \end{aligned}$$

  (140)
  $$\begin{aligned} |\varphi (d,u,x)|\le & {} B \Vert u\Vert _{\infty } + \gamma _2 |x| + b_2 |d|. \end{aligned}$$

  (141)

Let $c>0$² be a given constant and $a \in C^{0}([0,1])$ be a given function. Consider the mappings as $F:{\mathbb {R}}^n \times C^{0}([0,1]) \times {\mathbb {R}}^m \rightarrow {\mathbb {R}}^n$, $g:[0,1] \times {\mathbb {R}}^n \times C^{0}([0,1]) \rightarrow {\mathbb {R}}$, $\varphi : {\mathbb {R}}^q \times C^{0}([0,1]) \times {\mathbb {R}}^n \rightarrow {\mathbb {R}}$ being continuous mappings with $F(0,0,0)=0$ for which there exist constants $L>0$, ${\bar{N}}\in {[0,1 [}$ such that the inequalities $\max _{0\le z \le 1}(|g(z,x,u)-g(z,y,w)|)+|F(x,u,v)-F(y,w,v)|\le L|x-y|+L\Vert u-w\Vert _{\infty }$, $|\varphi (d,u,x)-\varphi (d,w,y)|\le \bar{N}|x-y|+\bar{N}\Vert u-w\Vert _{\infty }$, hold for all $u, w \in C^{0}([0,1])$, $x,y \in {\mathbb {R}}^n$, $v\in {\mathbb {R}}^{m}$, $d\in {\mathbb {R}}^{q}$. Suppose that Assumptions (H1) and (H2) hold and that the following small-gain condition is satisfied:

$$\begin{aligned} (\gamma _1 \gamma _3+ & {} A)c^{-1} \max _{0\le z \le 1}\left( p(z) \int _{0}^{z}\frac{1}{p(l)}dl \right) +(\gamma _2 \gamma _3+B)\max _{0\le z \le 1}(p(z)) \nonumber \\+ & {} 2 \sqrt{(\gamma _1 \gamma _3 +A)c^{-1}(\gamma _2 \gamma _3+B)\max _{0\le z \le 1}(p(z)) \max _{0\le z \le 1}\left( p(z)\int _{0}^{z}\frac{1}{p(l)}dl \right) }< 1 \end{aligned}$$

(142)

with $p(z) {:=} \exp \left( c^{-1}\int _{0}^{z}a(w)dw\right)$ for $z \in [0,1]$ [recall (8.2.11) and (8.2.14)] in [31, Section 8.2]. Then, there exist constants $\delta , \varTheta , \gamma > 0$ such that for every $u_0 \in C^{0}([0,1])$, $x_0 \in {\mathbb {R}}^{n}$, $d \in C^{0}({\mathbb {R}}_{+}\,;{\mathbb {R}}^{q})$ with $u_0(0) = \varphi (d(0),u_0,x_0)$, $f \in C^{0}([0,1] \times {\mathbb {R}}_{+})$, and $v \in C^{0}({\mathbb {R}}_{+}\,;{\mathbb {R}}^{m})$ the unique generalized solution of the initial-boundary value problem (136), (137), (138) satisfies the following estimate:

$$\begin{aligned} |x(t)|+\Vert u(t)\Vert _{\infty }\le & {} \varTheta (|x_0|+\Vert u_0\Vert _{\infty })\exp (-\delta t) \nonumber \\+ & {} \gamma \left[ \max _{0\le s\le t}(|v(s)|) + \max _{0\le s\le t}(\Vert f(s)\Vert _{\infty }) + \max _{0\le s\le t}(|d(s)|) \right] , \quad \forall t \ge 0. \end{aligned}$$

(143)