Recent development in computational complexity characterization of Nash equilibrium

Abstract

The computation of Nash equilibria has been a problem that spanned half a century that has attracted Economists, Operations Researchers, and most recently, Computer Scientists. The study of its complexity, in particular that of the two-player game, has come to a conclusion recently. It is, however, impossible without the subsuming ideas from important progresses made in the various fronts of its investigation. In this article, we present a review of the most relevant ideas, methods, and results in a way that would lead interested readers to get a full picture of the subject. We will also discuss some new issues opened up by the characterization of complexity for the two-player Nash equilibrium problem.

Introduction

The concept of Nash equilibrium, named after John Forbes Nash, is arguably the most influential solution concept in game theory. It considers a set of two or more players. Each of them has a finite number of actions to choose from, called pure strategies. If every player fixes its action, a payoff (potentially different) is determined for each of them. In the case of two-player games, the payoff functions, one for each of the two players, form two matrices whose rows and columns are indexed by the actions of the two players, respectively. Such a representation is alternatively called a bimatrix game where the players are often referred to as a row player and a column player.

Example 1

An example of two-player games is the Rock–Paper–Scissors game. In the game, the strategy set of each player is $\{\mathrm{rock},\mathrm{paper},\mathrm{scissors}\}$. When playing the game, each player needs to choose an action from the strategy set. Suppose the winner gets one dollar from the loser; and no one loses money when the game is tied. Then it can be described by the following table (Here we let $P_1$ and $P_2$ denote the two players, respectively.):

For example, the (1, 2)th entry of the table shows that if $P_1$ chooses rock and $P_2$ chooses paper, then $P_2$ wins one dollar from $P_1$. One can further decompose the table into a pair of matrices:

$$A=\left(\begin{array}{ccc}\hfill 0\hfill & \hfill -1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 0\hfill & \hfill -1\hfill \\ \hfill -1\hfill & \hfill 1\hfill & \hfill 0\hfill \end{array}\right)\text{and}B=\left(\begin{array}{ccc}\hfill 0\hfill & \hfill 1\hfill & \hfill -1\hfill \\ \hfill -1\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill -1\hfill & \hfill 0\hfill \end{array}\right)\text{,}$$

which are payoff matrices of the two players, respectively.

More generally, if the two players have $n$ and $m$ actions, respectively, to choose from, then the game can be represented by two $n\times m$ matrices.

A player’s decision on which action to take can be a random distribution over its strategy set, commonly referred to as a mixed strategy. Let ${\mathbb{P}}^n$ denote the set of all probability vectors in ${\mathbb{R}}^n$, i.e. non-negative, $n$-place vectors whose entries sum to 1. Then in the Rock–Paper–Scissors game, a mixed strategy of a player is a probability distribution $\{p_1,p_2,p_3\}\in {\mathbb{P}}^3$ over $\{\mathrm{rock},\mathrm{paper},\mathrm{scissors}\}$.

For a two-player game $(A,B)$, where $A$ and $B$ are both $n\times m$ matrices, a Nash equilibrium is a pair of mixed strategies, one for each player, such that no player can have a higher expected payoff by unilaterally changing its own mixed strategy. For simplicity of presentation, we normalize $A$ and $B$ such that all entries are between 0 and 1, i.e. in [0, 1]. Formally, a Nash equilibrium is a pair $(x^{\ast }\in {\mathbb{P}}^n,y^{\ast }\in {\mathbb{P}}^m)$ of mixed strategies, such that

$${\left(x^{\ast }\right)}^{T}A{y}^{\ast }\ge x^{T}A{y}^{\ast }\text{,}\forall x\in {\mathbb{P}}^n\text{and}{\left(x^{\ast }\right)}^{T}B{y}^{\ast }\ge {\left(x^{\ast }\right)}^{T}By\text{,}\forall y\in {\mathbb{P}}^m\text{.}$$

For the Rock–Paper–Scissors game, one can verify that there exists a unique Nash equilibrium: the mixed strategy for both players is $\{1/3,1/3,1/3\}$.

Computationally, one might settle with an approximate Nash equilibrium. A pair of mixed strategies $(x^o\in {\mathbb{P}}^n,y^o\in {\mathbb{P}}^m)$ is called an expected $ϵ$-approximate Nash equilibrium if

$${\left(x^o\right)}^{T}A{y}^o+ϵ\ge x^{T}A{y}^o\text{,}\forall x\in {\mathbb{P}}^n\text{and}{\left(x^o\right)}^{T}B{y}^o+ϵ\ge {\left(x^o\right)}^{T}By\text{,}\forall y\in {\mathbb{P}}^m\text{.}$$

This notaion of an expected $ϵ$-approximate Nash equilibrium is the most standard one and well known in the literature.

In this article, we focus on the related concept of $ϵ$-approximate Nash equilibrium [13] (also called $ϵ$-well-supported Nash equilibrium in [8]) for a two-player game $(A,B)$. It consists of a pair of mixed strategies $(x^{\ast }\in {\mathbb{P}}^n,y^{\ast }\in {\mathbb{P}}^m)$, such that

$$\forall 1\le j,k\le m\text{,}{\left(x^{\ast }\right)}^T{B}_j>{\left(x^{\ast }\right)}^T{B}_k+ϵ⟹y_k^{\ast }=0\text{and}\forall 1\le j,k\le n\text{,}{A}_j{y}^{\ast }>A_k{y}^{\ast }+ϵ⟹x_k^{\ast }=0\text{,}$$

where we use $A_j$ and $B_j$ to denote the $j$th row of $A$ and the $j$th column of $B$, respectively. In another word, only the pure strategies which are approximately optimal are played with non-zero probability (while in a Nash equilibrium, only the optimal ones are played with non-zero probability). This notion can be further extended to multi-player games.

From the definitions above, one can verify that every well-supported $ϵ$-approximate Nash equilibrium must also be an expected $ϵ$-approximate equilibrium, while the reverse is not true. We only know that an expected $ϵ$-approximate Nash equilibrium can be transformed, in polynomial time, into a well-supported $8nϵ$-approximate Nash equilibrium, as first shown in [8].

The idea of Nash equilibria can be traced back to the work of Antoine Augustin Cournot [12]. He developed a model called “duopoly”, a model of competitive markets, and mathematically derived an equilibrium solution. Then, Oskar Morgenstern and John von Neumann [36] studied two-player zero-sum games, where one player’s gain is another player’s loss under the same competitive equilibrium concept (e.g. the Rock–Paper–Scissors is of this type, since $A+B=0$). A proof of the existence of equilibria for two-player zero-sum games was established via von Neumann’s minimax theorem [57]. The most accessible proof is through the linear programming duality theorem, a special case of von Neumann’s minimax theorem. It also implies a polynomial-time solution by Leonid Khachiyan’s algorithm as we discuss later.

Once we have the duality theorem, the main idea of the linear programming formulation for the zero-sum game is quite simple. We first consider the case when the row player only uses pure strategies but the column player is allowed to use mixed strategies. Let $y\in {\mathbb{P}}^m$ be the mixed strategy of the column player. Once $y$ is fixed, the column player would argue that, the row player would choose a pure strategy that maximizes ${\left(x\right)}^{T}Ay$, that is, the row player would choose the largest item in the column vector $Ay$, equivalent to the value: $min\{v:Ay\le ve\}$, where $e$ is a vector of all ones.

As the payoff matrix of the column player is $B=-A$, its payoff is $-x^{T}Ay$. The column player wants to maximize it, and hence, to minimize $x^{T}Ay$ (with the negative sign removed). Therefore, it would like to have a vector $y\in {\mathbb{P}}^m$ such that the maximum item of the column vector $Ay$ is minimized.

Therefore, the goal of the column player is to find a mixed strategy $y$ to minimize $min\{v:Ay\le ve\}$. Similarly, if we consider the case when the column player is restricted to pure strategies but the row player can use mixed strategies, then the goal of the row player is to find $x\in {\mathbb{P}}^n$ that maximizes $max\{u:x^{T}A\ge u{e}^T\}$.

As the above two linear programs are dual to each other, they have exactly the same optimal value. Moreover, the optimal solutions $(x^{\ast },y^{\ast })$ turn out to be a Nash equilibrium of the game (in which we allow both players to choose mixed strategies simultaneously). This gives a linear program solution for zero-sum games, and therefore the latter is solvable in polynomial time.

Nash was the first to study the more general non-zero-sum games (e.g. the Prisoner’s dilemma) with two or more players along the competitive equilibrium approach, and he proved in [39] the existence of a competitive equilibrium under the general setting. Noticeably, Nash’s proof for non-zero-sum games took a different approach. It was based on the fixed point theorem of Luitzen Egbertus Jan Brouwer [3]: Every continuous map $f$ from ${[0,1]}^d$ to itself must have a fixed point $x^{\ast }\in {[0,1]}^d$ such that $f\left(x^{\ast }\right)=x^{\ast }$. Given any game, Nash constructed a Brouwer map whose fixed points are precisely equilibria of the game. As a result, the existence of equilibria followed as a result of the existence of fixed points. David Gale suggested to Nash an alternative, and conceptually much simpler, proof for the existence of Nash equilibria [38]. That proof was based on another fixed point theorem of Shizuo Kakutani [25], which is a generalization of Brouwer’s fixed point theorem. According to Gerard Debreu [17], Kakutani’s result also succinctly presents a closely related lemma of John von Neumann [56].

This line of techniques for proving the existence of equilibria has proven to be a powerful tool in the subsequent development of Mathematical Economics. A few years after Nash’s work, Kenneth J. Arrow and Gerard Debreu, derived the first rigorous proof for the existence of a market clearing equilibrium, under quite mild assumptions on the utility functions of market participants [2]. Again, their proof is based on the use of fixed point theorems. Arrow and Debreu’s result, has been regarded by many as one of the most beautiful applications of mathematical theories, as well as one of the most influential economic theories developed in the last century.

The developments in designing efficient algorithms for finding an equilibrium followed a similar course. First, since any two-player zero-sum game can be formulated as a linear program, one can use the simplex method of George Dantzig to find a Nash equilibrium in finite steps. Then, a decade after Nash’s work, Lemke and Howson developed a path-following, simplex-like algorithm for general two-player games [33]. Later, inspired by the path-following approach of Lemke and Howson, Herbert Scarf developed the first converging algorithm to compute fixed points [45], and to compute equilibrium prices of competitive markets [47]. Today, the path-following method has been recognized as one of the most important algorithmic paradigms in optimization.

Remember that every two player zero-sum game can be formulated as a pair of linear programs, each one dual to the other. For the non-zero-sum game case, it can be formulated as a linear complementary problem [11]. Assuming that, without loss of generality, both the row player and the column player have $n$ strategies, with payoff matrices $A$ and $B$, respectively. We define

$$U=\left(\begin{array}{cc}\hfill 0\hfill & \hfill A\hfill \\ \hfill B^T\hfill & \hfill 0\hfill \end{array}\right)\text{.}$$

Then, finding a two-player Nash equilibrium is equivalent to finding a non-trivial solution of the following linear complementary problem

$$H\left(\begin{array}{c}u\hfill \\ v\hfill \end{array}\right)=1\text{,}u\ge 0\text{,}v\ge 0\text{,}{u}^{T}v=0\text{,}$$

where $H=(I_{2n},U)$, and $I_{2n}$ is a $2n\times 2n$ identity matrix. The condition $u^{T}v=0$ is called the complementary condition. A solution $(u,v)$ to the above system corresponds to a Nash equilibrium (of the game) if and only if $(u,v)\ne (1,0)$. In such a case, we can write $v$ as $v={(x^T,y^T)}^T$ and $(x,y)$ is a Nash equilibrium of game $(A,B)$ (where $x$ is the row player’s strategy and $y$ is the column player’s strategy).

Starting from the trivial solution $(u,v)=(1,0)$, the algorithm of Lemke and Howson starts by increasing the value of $v_t$, for some index $t$. There is a limit on how much $v_t$ can be increased without violating the condition $u\ge 0$. At this limit $v_t^{\ast }$, some $u_j$ becomes zero. If $j=t$, we arrive at a non-trivial solution (and hence a Nash equilibrium), since the complementary condition still holds. Otherwise, (in the second step) $v_j$ can be increased without violating the condition $u_j{v}_j=0$. The Lemke–Howson’s algorithm continues, until it arrives at a non-trivial complementary solution. Under the condition that the matrix $U$ is obtained from a two-player game, Lemke–Howson’s algorithm converges in a finite number of steps. A nice property of the algorithm is that, at each intermediate step, either there is exactly one next move, or a non-trivial complementary solution is found. More detail about the algorithm can be found in Section 2.

Recall that in [29], Klee and Minty showed that for some specific instances of linear programming, the simplex method requires an exponential number of steps. In 1979, Khachiyan designed an ellipsoid algorithm that can solve a linear program in polynomial time [28]. The existence of such an algorithm implies that a Nash equilibrium in a two-player zero-sum game can be found in polynomial time. A natural immediate question is whether this result can be extended to the non-zero-sum two player games, by using a new algorithm different from the Lemke–Howson’s approach. When we look into the proofs of the existence results for two-player zero-sum games and general two-player games, we see a subtle difference between them. While the existence of an equilibrium in a two-player zero-sum game can be proved via linear programming duality, the proof for general games heavily relies on Brouwer’s fixed point theorem, the search version of which is not known to have a polynomial-time algorithm. Despite the considerable effort that has been devoted to the search of efficient algorithms for Nash’s problem, no polynomial-time algorithm has been found yet. Recently, Savani and von Stengel [44] have shown that for some specific two-player games, even, the algorithm of Lemke and Howson takes an exponential number of transitions of states.

The first step towards understanding the computational complexity of Nash equilibria started fifteen years ago when Christos Papadimitriou [41] defined the complexity class, Polynomial Parity Argument, Directed version (PPAD in short) to characterize mathematical proofs that rely on parity arguments. In Section 2, we will define formally the class PPAD. Such an argument appears in the proofs of many important theorems, including Sperner’s Lemma [53], which led to one of the most elegant proofs of Brouwer’s fixed point theorem. Therefore, it is not surprising that both a discrete version of the fixed-point problem and the Nash equilibrium problem are members of PPAD [41] (again more detail will be given in Section 2). The former is among the first problems shown to be PPAD-complete [41]. However, it is unknown whether the latter is also complete.

In 2005, an exciting breakthrough was made by Constantinos Daskalakis, Paul Goldberg and Christos Papadimitriou [13] when they proved that finding a Nash equilibrium is PPAD-hard for games with four or more players. Informally, their result states that to find an approximate equilibrium with an exponentially small error is PPAD-complete. It is known from Nash’s original work that there are three-player games which have no rational valued Nash equilibrium, therefore the approximation version is the only sensible computational way to approach the problem. Moreover, it also conforms to the idea of Bounded Rationality of Herbert Simon [50], which roughly states that we humans do not spend infinite amount of resources to achieve an ideal optimality in decision-making processes.

The four-player PPAD-completeness result was quickly extended to the three-player case [5], [14].

Let 2-Nash denote the problem of finding a Nash equilibrium in a two-player game. The results previously mentioned leave 2-Nash as the last open problem to classify according to its computational complexity. This fact was explicitly referred by Papadimitriou [40] as one of the two “most concrete open problems” at the boundary of P.

Compared to games for three or more players, finding a Nash equilibrium in a two-player game could be easier for several reasons. First, as mentioned previously, an equilibrium in a two-player zero-sum game can be found in polynomial time, by using any interior point algorithm. This fact raises a hope and a possibility that the Lemke–Howson algorithm could be someday replaced by a polynomial-time one. Recall, that a two-player game $(A,B)$ is rational, if all the entries of $A$ and $B$ are rational. Then the second reason pointing in the direction that 2-Nash could be easier is due to the fact that it is known that every rational two-player game has a rational equilibrium with only a polynomial number of bits. This property is not preserved in games among three or more players. Does this mathematical difference have any algorithmic implication?

On the other hand, the computational complexity of finding an equilibrium in a two-player game is particularly important. A hardness result for the two-player case could have significant implications to several other related problems. Most importantly, if 2-Nash is PPAD-complete, then, via a connection established by Yinyu Ye [59], one can show that the Arrow–Debreu general equilibrium problem is PPAD-hard.

Recently, we [7] proved that 2-Nash is indeed complete in PPAD. Later, this result was further strengthened by Shanghua Teng and ourselves [8] by showing that even the approximation of Nash equilibria is not any easier, and is still PPAD-complete. Together with [13], [5], [14], these results present a very clear characterization for the computational complexity of Nash equilibria.

In this survey, we plan to introduce the basic concepts, techniques and results that lead to the PPAD-completeness of the two-player Nash equilibrium problem. We will also discuss the subsequent results and interesting open problems.