A differentiable homotopy method to compute perfect equilibria

Abstract

The notion of perfect equilibrium was formulated by Selten (Int J Game Theory 4(1):25–55, 1975) as a strict refinement of Nash equilibrium. For an extensive-form game with perfect recall, every perfect equilibrium of its agent normal-form game yields a perfect equilibrium of the extensive-form game. This paper aims to develop a differentiable homotopy method for computing perfect equilibria of normal-form games. To accomplish this objective, we constitute an artificial game by introducing a continuously differentiable function of an extra variable. The artificial game defines a differentiable homotopy mapping and establishes the existence of a smooth path to a perfect equilibrium. For numerical comparison, we also describe a simplicial homotopy method. Numerical results show that the differentiable homotopy method is numerically stable and efficient and significantly outperforms the simplicial homotopy method especially when the problem is large.

Appendices

Appendix 1

This appendix proves that the Jacobian matrix $Dp(y,\mu ,t;\alpha )$ of $p(y,\mu ,t;\alpha )$ is of full-row rank for any $(y,\mu ,t;\alpha )\in {\mathbb {R}}^m\times {\mathbb {R}}^n\times (0,1]\times {\mathbb {R}}^m$. This property is used in the proof of Theorems 2 and 3.

Let $\varphi (y,t)=(u^i(s^i_j,x^{-i}(y,t)):i\in N,j\in M_i)^{\top }$. We have

$$\begin{aligned} D_y\varphi (y,t)=\left( \begin{array}{ccc} 2\max \{0, y^1_1\}\frac{\partial u^1(s^1_1,x^{-1}(y,t))}{\partial x^1_1} &{}\cdots &{} 2\max \{0, y^n_{m_n}\}\frac{\partial u^1(s^1_1,x^{-1}(y,t))}{\partial x^n_{m_n}}\\ 2\max \{0, y^1_1\}\frac{\partial u^1(s^1_2,x^{-1}(y,t))}{\partial x^1_1} &{}\cdots &{} 2\max \{0, y^n_{m_n}\}\frac{\partial u^1(s^1_2,x^{-1}(y,t))}{\partial x^n_{m_n}}\\ \vdots &{} \ddots &{} \vdots \\ \\ 2\max \{0, y^1_1\}\frac{\partial u^n(s^n_{m_n},x^{-n}(y,t))}{\partial x^1_1} &{}\cdots &{} 2\max \{0, y^n_{m_n}\} \frac{\partial u^n(s^n_{m_n},x^{-n}(y,t))}{\partial x^n_{m_n}} \end{array}\right) \end{aligned}$$

and

$$\begin{aligned} D_t\varphi (y,t)=\eta _0\left( \begin{array}{c} \sum \limits _{i\in N}\sum \limits _{j\in M_i}\frac{\partial u^1(s^1_1,x^{-1}(y,t))}{\partial x^i_j}\\ \sum \limits _{i\in N}\sum \limits _{j\in M_i}\frac{\partial u^1(s^1_2,x^{-1}(y,t))}{\partial x^i_j} \\ \vdots \\ \sum \limits _{i\in N}\sum \limits _{j\in M_i}\frac{\partial u^n(s^n_{m_n},x^{-n}(y,t))}{\partial x^i_j} \end{array}\right) . \end{aligned}$$

The Jacobian matrix $Dp(y, \mu , t; \alpha )$ of $p(y, \mu , t; \alpha )$ is given by

$$\begin{aligned} \begin{array}{rl} &{} Dp(y, \mu , t; \alpha )\\ &{}\quad = \left( \begin{array}{cccc} (1-\theta (t))D_y\varphi (y,t)+A &{}\quad -E^\top &{}\quad (1-\theta (t))D_t\varphi (y,t)+C &{}\quad -t(1-t)I\\ B &{}\quad 0 &{}\quad F &{}\quad 0 \end{array}\right) , \end{array} \end{aligned}$$

where I is an $m\times m$ identity matrix, $A= \begin{pmatrix} A_1\\ {} &{}\ddots \\ &{}&{}A_n \end{pmatrix}$ with

$$\begin{aligned} A_i=\begin{pmatrix} 2\min \{0, y^i_1\}-2\theta (t)\max \{0, y^i_1\}\\ {} &{}\ddots \\ &{}&{} 2\min \{0, y^i_{m_i}\}-2\theta (t)\max \{0, y^i_{m_i}\} \end{pmatrix}, \end{aligned}$$

$B= \begin{pmatrix} B_1^\top \\ {} &{}\ddots \\ &{}&{}B_n^\top \end{pmatrix}$ with $B_i= \begin{pmatrix} 2\max \{0,y^i_1\}\\ \vdots \\ 2\max \{0, y^i_{m_i}\} \end{pmatrix}$, $C=\begin{pmatrix}C_1\\ \vdots \\ C_n\end{pmatrix}$ with

$$\begin{aligned} C_i=-\theta (t)\eta _0e_{i}-\frac{d\theta (t)}{dt}\begin{pmatrix}u^i(s^i_1, x^{-i}(y,t))+x^i_1(y,t)-x^{0i}_1\\ \vdots \\ u^i(s^i_{m_i}, x^{-i}(y,t))+x^i_{m_i}(y,t)-x^{0i}_{m_i}\end{pmatrix} -(1-2t)\begin{pmatrix}\alpha ^i_1\\ \vdots \\ \alpha ^i_{m_i}\end{pmatrix}, \end{aligned}$$

$E=\begin{pmatrix} e_1^\top \\ {} &{}\ddots \\ &{}&{}e_n^\top \end{pmatrix}$ with $e_i=(1,1,\cdots ,1)^\top \in {\mathbb {R}}^{m_i}$, and $F=\eta _0(1,1,\ldots ,1)^{\top }\in {\mathbb {R}}^n$. Thus, for any $t\in (0,1)$, $Dp(y,\mu ,t;\alpha )$ is of full-row rank.

When $t=1$,

$$\begin{aligned} Dp(y, \mu , t; \alpha )= \begin{pmatrix} A &{}\quad -E^\top &{}\quad C &{}\quad 0\\ B &{}\quad 0 &{}\quad F &{}\quad 0 \end{pmatrix}. \end{aligned}$$

With Corollary 2, we have $y^{*i}_j=(x^{0i}_j-\eta _0)^{1/2}$ as $t=1$. Thus, A is invertible. Applying row operations, one can easily reduce $Dp(y, \mu , 1;\alpha )$ to

$$\begin{aligned} \begin{pmatrix} A &{}\quad -E^\top &{}\quad C &{}\quad 0\\ 0 &{}\quad BA^{-1}E^\top &{}\quad F-BA^{-1}C &{}\quad 0 \end{pmatrix}. \end{aligned}$$

Note that

$$\begin{aligned} BA^{-1}E^\top= & {} \begin{pmatrix} \sum \limits _{j=1}^{m_1}\frac{2\max \{0,y^1_j\}}{2\min \{0, y^1_j\}-2\max \{0, y^1_j\}}\\ &{}\ddots \\ &{}&{}\sum \limits _{j=1}^{m_n}\frac{2\max \{0,y^n_j\}}{2\min \{0, y^n_j\}-2\max \{0, y^n_j\}} \end{pmatrix}\\= & {} -\begin{pmatrix} m_1\\ &{}\ddots \\ &{}&{}m_n \end{pmatrix}. \end{aligned}$$

Thus, $Dp(y,\mu ,1;\alpha )$ is of full-row rank.

Appendix 2

The predictor-corrector method in this paper is as follows. We first parameterize $(y,\mu ,t)$ with the path length $\xi$ so that $y=y(\xi )$, $\mu =\mu (\xi )$ and $t=t(\xi )$. Let $Dp_\alpha (y(\xi ),\mu (\xi ),t(\xi ))$ denote the Jacobian matrix of $p_\alpha (y(\xi ),\mu (\xi ),t(\xi ))$. Then, consider the initial-value problem given by

$$\begin{aligned}&Dp_\alpha (y(\xi ),\mu (\xi ),t(\xi ))\left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right) ^{\top }=0,\\&\left\| \left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right) \right\| _2=1,\\&\text{ sign }\left( \det \left( \begin{array}{c}Dp_\alpha (y(\xi ),\mu (\xi ),t(\xi ))\\ \left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right) \end{array}\right) \right) =b_0,\\&y^{i}(0)=(x^{0i}-\eta _0e_i)^{1/2},\;\mu _i(0)=0,\;t(0)=1,\;i\in N, \end{aligned}$$

where $e_i=(1,1,\cdots ,1)^\top \in {\mathbb {R}}^{m_i}$ and $b_0$ is a number in $\{-1,1\}$ given by

$$\begin{aligned} b_0=\text{ sign }\left( \det \left( \begin{array}{c}Dp_\alpha ( y(0),\mu (0),t(0))\\ \left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right) \end{array}\right) \right) \end{aligned}$$

with $\left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right)$ being the solution of

$$\begin{aligned}&Dp_\alpha (y(0),\mu (0),t(0))\left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right) ^\top =0,\\&\left\| \left( \frac{dy(\xi )}{d\xi },\frac{d\mu (\xi )}{d\xi }, \frac{dt(\xi )}{d\xi }\right) \right\| _2=1,\\&\frac{dt(0)}{d\xi }<0. \end{aligned}$$

Let $Dp_\alpha (\xi )$ stand for $Dp_\alpha (y(\xi ),\mu (\xi ),t(\xi ))$, and $g(Dp_\alpha (\xi ))$ be the solution of

$$\begin{aligned}&Dp_\alpha (\xi )g=0,\\&\Vert g\Vert _2=1,\\&\text{ sign }\left( \det \left( \begin{array}{c}Dp_\alpha (\xi )\\ g^\top \end{array}\right) \right) =b_0. \end{aligned}$$

Then,

$$\begin{aligned} g(Dp_\alpha (\xi ))= \frac{(I-Dp_\alpha (\xi )^{\top }(Dp_\alpha (\xi ) Dp_\alpha (\xi )^{\top })^{-1}Dp_\alpha (\xi ))q}{\left\| (I-Dp_\alpha (\xi )^{\top }(Dp_\alpha (\xi ) Dp_\alpha (\xi )^{\top })^{-1}Dp_\alpha (\xi ))q\right\| _2}, \end{aligned}$$

where I is a $(m+n+1)\times (m+n+1)$ identity matrix and q is any given vector of ${\mathbb {R}}^{m+n+1}$ satisfying that

$$\begin{aligned} \text{ sign }\left( \det \left( \begin{array}{c}Dp_\alpha (\xi )\\ g(Dp_\alpha (\xi ))^\top \end{array}\right) \right) =b_0. \end{aligned}$$

To solve the above initial value problem, we adopt a predictor-corrector method from Allgower and Georg [1], which is as follows.

Initialization:

Let $y^{i}(0)=(x^{0i}-\eta _0e_i)^{1/2},\;\mu _i(0)=0$ for $i\in N$. Let $y(0)=(y^1(0),\ldots ,y^n(0)),\;\mu (0)=(\mu _1(0),\ldots ,\mu _n(0))$, and $t(0)=1$. Compute $Dp_\alpha (y(0),\mu (0),t(0))$. Let $Dp_\alpha ^0$ stand for $Dp_\alpha (y(0),\mu (0),t(0))$. Choose an arbitrary vector q of ${\mathbb {R}}^{m+n+1}$ satisfying that

$$\begin{aligned} \left( I-Dp_\alpha ^{0\top }(Dp_\alpha ^0 Dp_\alpha ^{0\top })^{-1}Dp_\alpha ^0\right) q\ne 0. \end{aligned}$$

The initial tangent vector is given by

$$\begin{aligned} g=\frac{\left( I-Dp_\alpha ^{0\top }(Dp_\alpha ^0 Dp_\alpha ^{0\top })^{-1}Dp_\alpha ^0\right) q}{\left\| \left( I-Dp_\alpha ^{0\top }(Dp_\alpha ^0 Dp_\alpha ^{0\top })^{-1}Dp_\alpha ^0\right) q\right\| _2}. \end{aligned}$$

If $g_{m+n+1}>0$, let $g=-g$. Compute

$$\begin{aligned} b_0=\text{ sign }\left( \det \left( \begin{array}{c}Dp_\alpha ^0\\ g^\top \end{array}\right) \right) . \end{aligned}$$

Let $\epsilon$ be a given tolerance and $\delta$ a sufficiently small positive number. Let $k=0$ and go to Step 1.

Step 1:

Choose a predictor-step length ${\bar{\beta }}>0$ satisfying that $\left\| p_\alpha ({\bar{y}},{\bar{\mu }},{\bar{t}})\right\| _{2}<\delta$ and $0<{\bar{t}}<1$, where

$$\begin{aligned} ({\bar{y}},{\bar{\mu }},{\bar{t}})^\top =(y_k,\mu _k,t_k)^\top +{\bar{\beta }} g. \end{aligned}$$

Solve the following system of nonlinear equations,

$$\begin{aligned} p_\alpha (y,\mu ,t)= & {} 0,\\ (y,\mu ,t)g= & {} ({\bar{y}},{\bar{\mu }},{\bar{t}})g, \end{aligned}$$

using Newton’s method with $({\bar{y}},{\bar{\mu }},{\bar{t}})$ being the starting point. Let $(y_{k+1},\mu _{k+1},t_{k+1})$ be an approximate solution of the system and $k=k+1$. Go to Step 2.

Step 2:

If $t_{k+1}<\epsilon$, then the method terminates and an approximate solution has been found. Otherwise, proceed as follows. Compute $Dp_\alpha (y_k,\mu _k,t_k)$. Let $Dp_\alpha ^{(k)}$ stand for $Dp_\alpha (y_k,\mu _k,t_k)$. Choose an arbitrary vector q of ${\mathbb {R}}^{m+n+1}$ satisfying that

$$\begin{aligned} \left( I-Dp_\alpha ^{(k)\top }(Dp_\alpha ^{(k)} Dp_\alpha ^{(k)\top })^{-1}Dp_\alpha ^{(k)}\right) q \ne 0. \end{aligned}$$

Let

$$\begin{aligned} g=\frac{\left( I-Dp_\alpha ^{(k)\top }(Dp_\alpha ^{(k)} Dp_\alpha ^{(k)\top })^{-1}Dp_\alpha ^{(k)}\right) q}{\left\| \left( I-Dp_\alpha ^{(k)\top }(Dp_\alpha ^{(k)} Dp_\alpha ^{(k)\top })^{-1}Dp_\alpha ^{(k)}\right) q\right\| _2}. \end{aligned}$$

Compute

$$\begin{aligned} b=\text{ sign }\left( \det \left( \begin{array}{c}Dp_\alpha ^{(k)}\\ g^{\top } \end{array}\right) \right) . \end{aligned}$$

If $b\ne b_0$, let $g=-g$. Go to Step 1. $\square$

The method is a standard predictor-corrector method. Its convergence analysis is referred to Allgower and Georg [1]. As mentioned in Sect. 2, the path leads to some point $(y^*,\mu ^*,0)$ on the target level $t=0$ such that $p_\alpha (y^*,\mu ^*,0)=0$.