Privacy, Patience, and Protection

Abstract

We analyze repeated games in which players have private information about their levels of patience and in which they would like to maintain the privacy of this information vis-à-vis third parties. We show that privacy protection in the form of shielding players’ actions from outside observers is harmful, as it limits and sometimes eliminates the possibility of attaining Pareto-optimal payoffs.

Appendices

Appendix

Proofs from Sect. 4

Proof of Lemma 1

In any equilibrium $\sigma$ and any history h reached with positive probability by $\sigma$, the corresponding mixed actions at h must belong to $B_i'$. Thus, the payoff profile at h belongs to $B_i$. Any infinite realized stream of payoffs to either player j, namely $\{u_j^k\}_{k=1}^\infty$, is such that $(u_i^k,u_{-i}^k)\in B_i$, and so $\lim _{T\rightarrow \infty }\frac{1}{T}\sum _{k=1}^T (u_i^k,u_{-i}^k) \in B_i$.¹⁶ Since $U_i(\sigma :L)$ and $U_{-i}(\sigma :S)$ are convex combinations of such limits, they also lie in $B_i$. $\square$

Proof of Lemma 2

In any equilibrium $\sigma$ and any history h reached with positive probability by $\sigma$, the corresponding mixed actions at h must be a NE of G. Thus, the payoff profile at h belongs to $V^{2S}$. Any infinite realized stream of payoffs to either player j, namely $\{u_j^k\}_{k=1}^\infty$, is such that $(u_i^k,u_{-i}^k)\in V^{2S}$, and so $\lim _{T\rightarrow \infty }\frac{1}{T}\sum _{k=1}^T (u_i^k,u_{-i}^k) \in V^{2S}$. Since $U_i(\sigma :L)$ and $U_{-i}(\sigma :S)$ are convex combinations of such limits, they also lie in $V^{2S}$. $\square$

A geometric interpretation of $\mathrm {PF}$ will be useful for later. Consider a plot of $V^*$, where player i’s payoffs are on the horizontal axis and player $-i$’s on the vertical axis. $V^*$ is a convex set, whose left and bottom boundaries correspond to the minimax payoffs of players i and $-i$, respectively. Then $\mathrm {PF}$ consists of the “top right” boundary of $V^*$, a connected set of line segments of nonpositive slope. Denote by $e^{-i}$ the top left endpoint of $\mathrm {PF}$ (where player $-i$ obtains the highest feasible utility), and by $e^{i}$ the bottom right endpoint (where player i obtains the highest feasible utility). If $\mathrm {PF}$ is a singleton, then $\mathrm {PF}=\{e^i\}=\{e^{-i}\}$. If $\mathrm {PF}$ is a line segment, it connects $e^{-i}$ to $e^{i}$. Otherwise, $\mathrm {PF}$ consists of connected line segments: starting at $e^{-i}$, proceeding toward some vertex $v_1$, then proceeding to another vertex $v_2$, and so on, until $e^{i}$.

Note that the vertices $v_k$ of $\mathrm {PF}$ correspond to payoffs of pure action profiles of the stage game G. The endpoints $e^i$ and $e^{-i}$, on the other hand, could correspond to payoffs of pure action profiles or could be convex combinations of payoffs of pure action profiles. In the latter case, it must be that one of the pure action profiles in the convex combination is not IR for one of the players.

Proof of Lemma 3

Any element of $V^{2S}$ is a convex combination of the payoffs of NE of G. Since by assumption the Pareto frontier is not defined by the NE of G, there exists some v on the Pareto frontier that is not a convex combination of payoffs of NE of G. Thus, $v\not \in V^{2S}$. $\square$

Proof of Lemma 4

Let $F=\{e^{-i},v^1,\ldots ,v^k,e^{i}\}$ denote the set of endpoints and vertices along $\mathrm {PF}$, and suppose toward a contradiction that all of $\mathrm {PF}$ lies in $B_i$. Consider the payoff profiles F that define the Pareto frontier, and note that, by assumption, not all are NE payoffs of the stage game. So there is some payoff profile $x\in F$ that cannot be attained as the payoff pair of a NE of G. Furthermore, because $x\in \mathrm {PF}$ and $\mathrm {PF}\subseteq B_i$ it follows that $x\in B_i$.

We now consider two cases: that $x\in V_p$, the set of feasible points attainable by a pure strategy profile, and that it is not. In the former case, x is a payoff profile that can only be attained as the payoff of some pure action profile $a^x=(a^x_i, a^x_{-i})$, namely $x=u(a^x)$. Furthermore, since $x\in B_i$ it must be the case that $a^x\in B_{i}'$, and so

$$\begin{aligned} a^x_{-i}\in \mathrm {BR}(a^x_i) . \end{aligned}$$

(1)

Consider now the payoff profile ${{\overline{x}}}=(x_{-i},x_i)$. By symmetry, ${{\overline{x}}}\in V_p$ is a payoff profile that lies in $\mathrm {PF}$ that can only be attained as the payoff of the pure action profile $a^{{\overline{x}}}=(a^{{\overline{x}}}_i, a^{{\overline{x}}}_{-i})=(a^x_{-i},a^x_i)$. Furthermore, since ${{\overline{x}}}\in \mathrm {PF}$ it must be the case that ${{\overline{x}}}\in B_i$. This implies that $a^{{\overline{x}}}\in B_{i}'$, and so $a^{{\overline{x}}}_{-i}\in \mathrm {BR}(a^{{\overline{x}}}_i)$, and thus

$$\begin{aligned} a^x_{i}\in \mathrm {BR}(a^x_{-i}) . \end{aligned}$$

(2)

Combining (1) and (2) implies that $a^x$ is a NE of the game, contradicting the assumption that x cannot be attained as the payoff profile of a NE.

We now consider the second case, that x is not an element of $V_p$. In particular, this means that $x\in \{e^i, e^{-i}\}$, since the other elements of F are all vertices of V and so lie in $V_p$.

Now, x is not the payoff of a pure action profile but can be attained as the convex combination of the payoffs of two pure action profiles. Denote these payoff profiles by y and z, where $y=u(a^y)$ and $z=u(a^z)$. By genericity assumption 2, x can only be attained as the convex combination of these two pure profiles (and no other pure profiles). Furthermore, one element of $\{y,z\}$ is in $\mathrm {PF}$, and the other is not in $\mathrm {PF}$, and thus is not IR for one of the players. Let z be the former and y the latter.

Because the game is symmetric we can assume without loss of generality that $x=e^{-i}$. By symmetry, the point ${{\overline{x}}}=e^{i}$ is also in $\mathrm {PF}$, and furthermore, ${{\overline{x}}}$ is the convex combination of ${{\overline{y}}}$ and ${{\overline{z}}}$, where ${{\overline{y}}}=(y_{-i},y_i)$ and ${{\overline{z}}}=(z_{-i},z_i)$. Note that z and ${{\overline{z}}}$ are pure profiles in $\mathrm {PF}$, that y is not IR for player i, and that ${{\overline{y}}}$ is not IR for player $-i$. As there is no point in $B_{i}'$ in which $-i$ does not best respond to some action of player i, the payoff profile ${{\overline{y}}}\not \in B_{i}$, nor is any payoff of player $-i$ that is below ${{\overline{x}}}_{-i}$: These are not IR for player $-i$. Thus, the only way ${{\overline{x}}}\in B_i$ is if the mixed profile $a^{{\overline{x}}}$ satisfies $a^{{\overline{x}}}_{-i}\in \mathrm {BR}(a^{{\overline{x}}}_i)$. However, since ${{\overline{x}}}$ is on the border (and not the interior) of $V^*$, genericity assumption 1 implies that $a^{{\overline{x}}}$ is such that only one player mixes. Furthermore, since player $-i$ is short-sighted, and one realization of the mixture yields a payoff below $-i$’s IR payoff, it must be player i who mixes. To conclude, the payoff profile ${{\overline{x}}}$ can only be obtained as the payoff of some mixed action profile $a^{{\overline{x}}}$ in which only player i mixes between two actions, say actions b and c. Thus, $a^{{\overline{y}}}=(b,a^{{\overline{x}}}_{-i})$ and $a^{{\overline{z}}}=(c,a^{{\overline{x}}}_{-i})$, such that $a^{{\overline{x}}}_{-i}$ is a best response to the mixture $a^{{\overline{x}}}_{i}$ of b and c.

Now consider the payoff profile $x=e^{-i}$. Recall that x is the convex combination of y and z, each of which is the utility profile of a unique pure action profile, say $a^y$ and $a^z$. By symmetry, $a^y=(a^x_{i},b)$ and $a^z=(a^x_{i},c)$, both of which are pure action profiles. Furthermore, $a^y,a^z\in B_{i}'$, and so both $b,c\in \mathrm {BR}(a^x_i)$. But note that $a^x_i=a^{{\overline{x}}}_{-i}$. This, together with the above, implies that $a^x$ and $a^{{\overline{x}}}$ are NE of G, which is a contradiction. $\square$

Proof of Lemma 5

The set of NE of G that are in $\mathrm {PF}$ is empty, and so the intersection of $\mathrm {CO}(\mathrm {NE}(G))$ with $\mathrm {PF}$ is empty. Since $V^{2S} = \mathrm {CO}(\mathrm {NE}(G))$, it follows that $V^{2S} \cap \mathrm {PF}= \emptyset$. $\square$

Proof of Lemma 6

For the realized LL case, this is almost immediate: The only limitation on $V^*$ is that payoffs be IR, and clearly no equilibrium of the repeated game with incomplete information leads to payoffs that are not IR.

Next, fix any equilibrium profile $\sigma$. Let $\sigma _i^L$ denote the strategy of player i of type L, and $\sigma _i^S$ the strategy of player i of type S. Furthermore, denote the belief of player i about player $-i$’s strategy as $\sigma ^{-i}$. Note that initially $\sigma ^{-i}$ places weight $\beta _i$ on $\sigma _{-i}^S$ and weight $1-\beta _i$ on $\sigma _{-i}^L$. Thus, $\sigma ^{-i}$ is absolutely continuous with respect to both $\sigma _{-i}^L$ and $\sigma _{-i}^S$. By Theorem 1 of Kalai and Lehrer [34], for every $\varepsilon >0$ and almost every play path h, there is a time $T=T(h,\varepsilon )$ after which the strategy of the realized type of player $-i$ is $\varepsilon$-close to the belief $\sigma ^{-i}$ that player i has about his opponent’s strategy.

First, suppose the realized types are L and S. From some point on the S player i best responds to his beliefs $\sigma ^{-i}(h)$, and his beliefs are $\varepsilon$-close to the actions of the opponent. That is, from some point on the strategy profile at stage t is $\sigma _i^S(h^t)\in \mathrm {BR}(\sigma ^{-i}(h^t))$, where $\sigma ^{-i}(h^t)$ is $\varepsilon$-close to $\sigma _{-i}^L(h^t)$. This implies that i’s payoff under $\sigma _i^S(h^t)$ is $\varepsilon$-close to his payoff in some profile in $V^{LS}_i$. The long-sighted player’s payoff is also in $V^{LS}_i$, since at every stage t his opponent is best responding to $\sigma ^{-i}(h^t)$, which is a possible mixed action of player $-i$.

Next, suppose the realized types are S and S. Note that in $\sigma$, players may not be playing a NE of the stage game in early stages, since they are best responding to their beliefs, which place positive weight on the other player being of type L. At every history h, player i is best responding to his belief $\sigma ^{-i}(H)$ about the other’s mixed action at that history. Furthermore, from stage T onward, the behavior of the other player is $\varepsilon$-close to $\sigma ^{-i}$. From that point on, each player i is best responding to beliefs that are $\varepsilon$-close to the true strategy of the opponent. We claim that from this stage on their payoffs in each stage must be close to the payoffs of some NE of G, which will imply that the limit-of-means payoff of the interaction is in $V^{2S}$.

To see this, consider some sequence of histories $\left( h^t\right) _{t\ge T}$. We claim that for every $\delta >0$ and sufficiently large t, the payoff profile $u(\sigma _i^S(h^t),\sigma _{-i}^S(h^t))$ is within $\delta$ of $u(\alpha )$, where $\alpha$ is the convex combination of some NE of G. In fact, we claim something stronger: that all the partial limits of $\left( \sigma _i^S(h^t),\sigma _{-i}^S(h^t)\right) _{t\ge T}$ are NE of G.

For suppose toward a contradiction that this is false. Then there exists some partial limit $(x_i,x_{-i})$, some $x_i'\in \Delta (A_i)$, and some $\gamma >0$ such that $u_i(x_i',x_{-i})>u_i(x_i,x_{-i})+\gamma$. Let $(x_i^t,x_{-i}^t)_t$ be the subsequence of $\left( \sigma _i^S(h^t),\sigma _{-i}^S(h^t)\right) _{t\ge T}$ that converges to $(x_i,x_{-i})$. For all sufficiently large t, Theorem 1 of Kalai and Lehrer [34] implies that $x_i^t\in \mathrm {BR}(y_i^t)$, where $y_i^t$ is $\frac{\delta }{2(\max _{a} u_i(a)-\min _a u_i(a))}$-close to $x_{-i}^t$. But this implies that $x_i^t$ is a $\delta /2$-best response to $x_{-i}^t$. This contradicts the existence of the $x_i'$ above. Thus, all the limit points are NE, and so the limit-of-means payoff is in the convex combination of all NE payoffs, namely $V^{2S}$. $\square$

Equilibrium Refinements

In this section, we consider strengthening Theorem 1 with a refinement of perfect perception equilibrium. One possibility would be to consider perfect perception equilibria that satisfy the intuitive criterion of Cho and Kreps [15]. Gradwohl and Smorodinsky [29] define a variant of the intuitive criterion that applies to perception games, and this definition is easily amenable to our repeated setting.

In our setting, the intuitive criterion can be described as follows. Fix a perfect perception equilibrium $\sigma$ and a history h reached with positive probability by $\sigma$. Consider a player i, type $t_i$, and a deviation by this type at h to an action that has probability 0 under $\sigma$, and let the perception following that action place weight 1 on type $t_i$. Then if type $t_i$ strictly gains from this deviation, whereas type $t_{-i}$ weakly prefers the equilibrium strategy, then $\sigma$ does not satisfy the intuitive criterion.

It is straightforward to show that the perfect perception equilibrium construction of Theorem 1 satisfies the intuitive criterion. However, it does so in an uninteresting way, as the definition is vacuously true: No player will ever gain from a deviation that leads to a perception that places weight 1 on his type, because of the associated privacy cost. Thus, in no perfect perception equilibrium will the intuitive criterion have any bite.

In the remainder of this section, we consider a modification of the intuitive criterion that does have some bite in our model, and then provide a strengthening of our theorem that utilizes this refinement. Roughly, for a given profile of strategies $\sigma$, we define the notion of $\sigma$-intuitive beliefs. We then show that for every $v\in V^*$ there is a $\sigma$ such that for all $\sigma$-intuitive beliefs $\tau$, the profile $(\sigma ,\tau )$ is a perfect perception equilibrium with payoff profile v.

We begin with a definition of our refinement, followed by the theorem and intuition.

Definition of refinement Denote by

$$\begin{aligned} U_i(L, \gamma _i, \sigma _{-i}, \tau _i)\mathbin {{\mathop {=}\limits ^\mathrm{def}}}U_i(\gamma _i,\sigma _{-i}:L) -E_{(\gamma _i,\sigma _{-i})}c_i(L,\beta _i,\tau _i(a^\infty )) \end{aligned}$$

the utility of a long-sighted player, when strategies are $(\gamma _i, \sigma _{-i})$ and beliefs are $\tau _i$. For any history h, denote by

$$\begin{aligned} U_i(S, \gamma _i, \sigma _{-i}, \tau _i,h)\mathbin {{\mathop {=}\limits ^\mathrm{def}}}E(u_i(\gamma _i(h),\sigma _{-i}(h))|t_i=S)- E_{(\gamma _i(h),\sigma _{-i}(h))}c_i(S,\tau _i(h),\tau _i((h,a))) \end{aligned}$$

the utility of a short-sighted player at h, when strategies are $(\gamma _i, \sigma _{-i})$ and beliefs are $\tau _i$.

The following refinement restricts BB’s beliefs on profiles for which players assign probability zero. In particular, it states that if one type of player will not gain from deviating regardless of the beliefs at a deviation, whereas another type will gain by deviating given some belief, then the support of BB’s belief must consist only of the latter type.

Definition 2

For a given strategy profile $\sigma$, beliefs $\tau$ are $\sigma$-intuitive if they are rational w.r.t. $\sigma$, and if they satisfy the following for every $i\in \{1,2\}$, $h\in \cup _{k=0}^\infty A^k$, and strategy $\gamma _i$ that is identical to $\sigma _i$ everywhere except at histories that have h as a prefix,:

• If

  • 1. $U_i(L, \sigma , \tau _i)\ge U_i(L, \gamma _i, \sigma _{-i}, \tau _i')$ for every $\tau _i'$ for which $\tau _i(h')=\tau _i'(h')$ at every $h'$ that is reached with positive probability under $\sigma$;

  • 2. $U_i(S, \sigma , \tau _i,h)\le U_i(S, \gamma _i, \sigma _{-i}, \tau _i',h)$ for some $\tau _i'$ for which $\tau _i(h')=\tau _i'(h')$ at every $h'$ that is reached with positive probability under $\sigma$; and

  • 3. at least one of the inequalities above is strict,

  then $\tau _i(h,a)=1$ for every $a\in \mathrm {supp}(\gamma _i(h))\setminus \mathrm {supp}(\sigma _i(h))$.

• If

  • 1. $U_i(S, \sigma , \tau _i,h)\ge U_i(S, \gamma _i, \sigma _{-i}, \tau _i',h)$ for every $\tau _i'$ for which $\tau _i(h')=\tau _i'(h')$ at every $h'$ that is reached with positive probability under $\sigma$;

  • 2. $U_i(L, \sigma , \tau _i)\le U_i(L, \gamma _i, \sigma _{-i}, \tau _i')$ for some $\tau _i'$ for which $\tau _i(h')=\tau _i'(h')$ at every $h'$ that is reached with positive probability under $\sigma$; and

  • 3. at least one of the inequalities above is strict,

  then $\tau _i(h,a)=0$ for every $a\in \mathrm {supp}(\gamma _i(h))\setminus \mathrm {supp}(\sigma _i(h))$.

Strengthening of Theorem 1 We are now ready to state our theorem.

Theorem 3

If $c_i(S,p,1)$ is sufficiently large, then for every $v \in V^*$ there exists a strategy profile $\sigma$ such that for all $\sigma$-intuitive beliefs $\tau$, the profile $(\sigma , \tau )$ is a perfect perception equilibrium in which the long-run payoff profile is v. Furthermore, there exists a $\sigma$-intuitive $\tau$.

Proof

Let $\sigma$ be the profile in which both types of both players play the standard grim trigger strategy leading to payoff v, as in Theorem 1. The L type will never gain by deviating, since this will lead to (long-term) punishment and a nonnegative privacy cost. For an S type, if at some history h he has a profitable deviation (for some belief), then the perception must place weight 1 on type S after that deviation at h—formally, $\tau _i(h')=1$ whenever h is a prefix of $h'$—and so the deviation will not be profitable for him with the given belief. If at some h the S type has no profitable deviation for any belief, then the refinement has no bite at that deviation, and so the perception at the deviation can be anything: there will be no profitable deviation since neither type gains from deviating at h, by assumption. $\square$

The Benefits of Privacy Protection

In Sect. 4, we argued that privacy protection is harmful, as it may hinder the ability of players to obtain Pareto-optimal payoff profiles. That is, when comparing the best payoffs attainable in equilibrium, privacy protection is harmful. In this section, we show that there may be benefits to privacy protection. In particular, we show that in some games privacy protection can prevent the players from obtaining suboptimal payoffs. More specifically, we show that when comparing the worst payoffs in equilibrium, privacy protection can be beneficial. We illustrate two distinct such benefits, both for the particular case in which the stage game G is the prisoner’s dilemma (PD) from Fig. 1a, or a small modification thereof (although it will be clear that the ideas extend to other games as well).

1.1 Avoiding Non-IR Payoffs

When there is no privacy protection, Theorem 1 states that all of $V^*$ can be obtained in equilibrium. However, it may also be possible to obtain lower payoffs for some player. For example, suppose G is the PD, and that both $c_1(S, \beta _i, 1)$ and $c_1(L, \beta _i, 1)$ are high. That is, both types of player 1 incur a high cost to the belief that they are short-sighted type (whether or not this is true).

Here, there is an equilibrium in which both types of player 1 always play C, while both types of player 2 always play D. This is not IR for player 1, and he obtains a low payoff of 0. However, the equilibrium can be sustained by perceptions $\tau _1(h)=1$ for all histories h that are not on the equilibrium path. Note that under privacy protection, such a low payoff to player 1 is impossible, as he will always obtain at least his minimax payoff of 1.

Two additional notes are in order. First, the “bad” equilibrium above does not satisfy the refinement of Definition 2. However, other (more complicated) equilibria can be constructed that do satisfy the refinement and which also lead to non-IR payoffs.

Second, equilibria with such low payoffs are not always possible, and their existence depends on the specifics of the privacy cost function. For example, in the PD, if only the short-sighted type incurs privacy costs—formally, if $c_i(L,\beta _i,0)\equiv 0$—then no player will ever get non-IR payoffs in equilibrium.

1.2 Higher Minimax Values

When there is no privacy protection, players get at least the minimax payoffs ${\underline{v}}_i=\min _{\alpha _{-i}} \max _{a_i} u_i(a_i, \alpha _{-i})$, regardless of whether their opponent is long-sighted or short-sighted. With privacy protection, however, a player i facing a short-sighted opponent has a different minimax payoff, namely ${\underline{v}}_i'=\min _{\alpha \in B_{i}} \max _{a_i} u_i(a_i, \alpha _{-i})$: this is the minimax value under the additional condition that the short-sighted player $-i$ is best responding to player i. In some games, ${\underline{v}}_i'>{\underline{v}}_i$. In such games, the worst-case (over all equilibria) payoff of a player without privacy protection will be lower than her worst-case payoff with privacy protection.

A simple example of a game in which ${\underline{v}}_i'>{\underline{v}}_i$ is the PD, but where each player has a third option B, to set off a bomb. Payoffs are such that if one player chooses B, both players get $-1000$, and if both players choose B, they both get $-1001$. In this game, the minimax payoff is ${\underline{v}}_i = -1000$, whereas the minimax payoff of a player facing a short-sighted player is ${\underline{v}}_i' = 1$. This is because choosing B is a strictly dominated action, and so an S type will never choose it. In particular, he cannot use it to threaten punishment on the opponent.

Construction

1.1 Preliminaries

LS: Suppose player i is long-sighted and player $-i$ is short-sighted. What are the possible payoffs? Recall that

$$\begin{aligned} B_{i}=\{(\alpha _1,\alpha _2):\alpha _{-i}\in \mathrm {BR}(\alpha _i)\} \end{aligned}$$

is the set of feasible mixed actions, and

$$\begin{aligned} {\underline{v}}_i'=\min _{\alpha \in B_{-i}} \max _{a_i} u_i(a_i, \alpha _{-i}) \end{aligned}$$

the minimax payoff of player i. Next, let

$$\begin{aligned} V_i'=\mathrm {CO}\{(v_i,v_{-i})\in {{\mathbb {R}}}^2:(v_i,v_{-i})=(u_i(\alpha ),u_{-i}(\alpha ))~\text{ for } \text{ some }~\alpha \in B_{-i}\}, \end{aligned}$$

and

$$\begin{aligned} V_i^{LS}=\{(v_i,v_{-i})\in V_i': v_i\ge {\underline{v}}_i'\}. \end{aligned}$$

When types are known, Fudenberg et al. show that all player i payoffs in $V_i^{LS}$, and only those, are attainable as equilibrium payoffs of the repeated game ([21], Proposition 5). Their proof can be extended to show that, in fact, all payoff pairs in $V_i^{LS}$ can be attained in equilibrium:

Lemma 7

Suppose player i is long-sighted and player $-i$ short-sighted. For every $v\in V_i^{LS}$, there exists an equilibrium strategy profile $\sigma$ such that the long-run average payoffs of the players are v.

Proof

Since $v\in V_i^{LS}$, there exist three mixed-action profiles $\alpha ^1, \alpha ^2, \alpha ^3 \in B_{-i}$ such that $v\in \mathrm {CO}\{u(\alpha ^1),u(\alpha ^2),u(\alpha ^3)\}$. Furthermore, by standard folk theorem arguments, there is an infinite sequence of alternations among the three mixed-action profiles, for which the long-run average payoffs are exactly v. Denote this sequence by $\{s^k\}_{k=1}^\infty$, where each $s^k\in \{\alpha ^1,\alpha ^2,\alpha ^3\}$.

Modify the construction of $\sigma$ from the proof of Fudenberg et al.’s Proposition 5 as follows: Whenever $I_t\le 0$, let $\sigma ^t$ be the first unplayed $s^k$ in the sequence. If $I_t>0$ let $\sigma ^t=m^i$, the profile that minimaxes player i.¹⁷ The remainder of the proof is the same as in Fudenberg et al. [21]. $\square$

2S: Suppose both players are short-sighted. Recall that

$$\begin{aligned} V^{2S}=\mathrm {CO}\{v: v=(u_1(\alpha ), u_2(\alpha ))~\text{ for } \text{ some } \text{ NE }~\alpha ~\text{ of }~G\}. \end{aligned}$$

When types are known, the long-run payoffs of the repeated game for the players are in $V^{2S}$. That is, for any equilibrium $\sigma$, both $U_i(\sigma :S)\in V^{2S}$. Then:

Lemma 8

Suppose both players are short-sighted. For every $v\in V^{2S}$, and only such v, there exists an equilibrium strategy profile $\sigma$ such that the long-run average payoffs of the players are v.

Proof

If $v\in V^{2S}$, then there exist three Nash equilibrium profiles $\alpha ^1, \alpha ^2, \alpha ^3$ such that $v\in \mathrm {CO}\{u(\alpha ^1),u(\alpha ^2),u(\alpha ^3)\}$. By standard folk theorem arguments, there is an infinite sequence of alternations among the three mixed-action profiles, for which the long-run average payoffs are exactly v. Let $\sigma$ be the strategy profile that alternates between these profiles in this manner.

Now suppose $v\not \in V^{2S}$, but that there is an equilibrium $\sigma$ with long-run payoffs equal to v. If under $\sigma$ in every stage of the game both players play a NE of G, then $v\in V^{2S}$. Thus, in some stage of the game players must play a non-NE action profile. This, however, cannot be an equilibrium for two short-sighted players. $\square$

1.2 The Construction

We have the following construction:

Theorem 4

For any $(v,v^{LS}_1,v^{LS}_2,v^{2S})\in V^*\times V_1^{LS}\times V_2^{LS}\times V^{2S}$, there exists an equilibrium $\sigma =\sigma (t_1,t_2)$ such that:

• if $t_1=t_2=L$, then $(U_1(L,\sigma ),U_2(L,\sigma ))=v$;

• if $t_i=L$ and $t_{-i}=S$, then $(U_i(L,\sigma ),{\overline{U}}_{-i}(S,\sigma ))=v_i^{LS}$; and

• if $t_1=t_2=S$, then $({\overline{U}}_1(S,\sigma ),{\overline{U}}_2(S,\sigma ))=v^{2S}$.

The proof is by construction of a strategy profile that has the following structure. First, the players play a series of stage games in which player 1 “reveals” his type to player 2. Then, they play a series of stage games in which player 2 “reveals” his type to player 1. Finally, the players play folk theorem strategies corresponding to their now commonly known types. The challenge lies in constructing the two revelation phases in such a way that they will be part of the equilibrium of the repeated game.

For each player i, we will consider three cases for the revelation phase. The first case is easiest and applies to stage games G in which player i has a dominant action. The second case applies when there is a NE of the stage game G in which player i plays a mixed action. Finally, the third and most involved case applies when neither of the first two cases does, namely, when there is no dominant action and, in all NE of G, player i plays a pure action.¹⁸

For each kind of revelation phase, we will argue that it can be part of an equilibrium of the repeated game. This requires that the S type of each player play a best response to the other player in every stage game. Additionally, the L type of each player must either best respond in a stage game or play a suboptimal action, but can do the latter only finitely many times. Finally, in order to obtain the claimed long-run payoff, the revelation phases must end with probability 1 in finitely many stages.

1.2.1 Dominant Action

Let a be a dominant action of player i. The revelation phase will last one round, in which the short-sighted type of player i plays action a, and the long-sighted type of player i plays some other action $b\ne a$. Both types of player $-i$ play the same action c that is a best response to the mixed action that plays a with probability $\pi$ and b with probability $1-\pi$, where $\pi$ is the probability that player i is the short-sighted type.

This clearly reveals player i’s type, as S and L play different actions. It is also a best response for the S type, since he plays a dominant action, and for both types of player $-i$, since they play a best response. Thus, this revelation phase can be part of an equilibrium of the repeated game.

1.2.2 Mixed NE Action

Suppose now that player i has no dominant action in G. Let $\alpha$ be a NE of G in which player i plays a mixed strategy, and let a be the action played by player i with minimal but positive probability in this equilibrium. Suppose $\alpha _i(a)=q$. Also, suppose that the probability that player i is the short-sighted type is $\pi$. The type-revelation phase for player i will consist of a sequence of stage games G, where both types of player $-i$ play $\alpha _{-i}$. The two types of player i play differently, as follows.

Note that this type-revelation phase can be part of an equilibrium of the repeated game, since both types of both players play actions that are part of a NE.

Furthermore, this phase leads to the revelation of player i’s type with probability 1. Suppose first that he is short-sighted. Whenever the players play (2) above, his type is revealed with probability $q/\pi$. When they play (1) above, his type will not be revealed, but the posterior on $\pi$ increases by a factor of 1/q. Within a finite number of stages, then, $\pi$ will once again be greater than q, and they will play (2) again, and so on. Thus, player i’s type will be revealed in a finite number of rounds, and this revelation phase will end. More formally, for any $\varepsilon >0$, there is a K such that after K repetitions, the probability that player i’s type will be revealed as S is at least $1-\varepsilon$.

Finally, a similar argument holds when player i is long-sighted: in that case he will fully reveal his type when players play (1), and they can play (2) at most a finite number of times for each time they play (1).

1.2.3 Pure NE Action only

The last case to consider is when player i has no dominant action in G and when all NE of G are such that i plays a pure action. This case is more involved, but the basic strategy will be to construct a type-revelation phase that lasts at most k rounds. In each stage game of the phase, both types of player $-i$ will best respond to player i, and the S type of player i will best respond to player $-i$. The L type of player i, however, will play a different action with some small probability, in order to allow for separation. The construction of such a strategy is closely related to the notion of a trembling-hand perfect equilibrium and requires some additional definitions.

An $\varepsilon$-mixed action is a mixed action that places weight at least $\varepsilon$ on each pure action. Furthermore, recall that a trembling-hand perfect equilibrium (THPE) $\alpha$ of a game G is a mixed-action profile such that there exists a sequence $(\varepsilon ^k)_{k\ge 0}$ that converges to 0 and a sequence $(\alpha ^k)_{k\ge 0}$ that converges to $\alpha$, and for which each $\alpha ^k$ is $\varepsilon ^k$-mixed, such that for each player i, the mixed action $\alpha _i$ is a best response to $\alpha _{-i}^k$ for all k.

Definition 3

A one-sided THPE for player i is a THPE where only player i trembles. Formally, it is a mixed-action profile, $\alpha$, such that there exists a sequence $(\varepsilon ^k)_{k\ge 0}$ that converges to 0 and a sequence $(\alpha _i^k)_{k\ge 0}$ that converges to $\alpha _i$ for which the following hold:

• each $\alpha _i^k$ is $\varepsilon ^k$-mixed;

• $\alpha _i$ is a best response to $\alpha _{-i}$; and

• $\alpha _{-i}$ is a best response to $\alpha _{i}^k$ for all k.

Note that any (one-sided) THPE is also a NE.

Lemma 9

In any game, there exists a one-sided THPE for player i.

Lemma 10

For any sequence $\{\varepsilon ^m\}_{m\ge 0}$ that converges to 0, there exists a convergent subsequence $\{\varepsilon ^k\}_{k\ge 0}$ and a one-sided THPE $\alpha$ for player i with a corresponding sequence $\{\alpha _i^k\}_{k\ge 0}$, such that each strategy $\alpha _i^k$ is $\varepsilon ^k$-mixed.

Proof

The proof is analogous to the standard proof for the existence of a THPE (see, e.g., Proposition 249.1 in Osborne and Rubinstein [43]), with the proper modifications for a one-sided THPE for player i. We include it here for completeness.

For each m, define the normal game $G_m$ to be the one in which player i’s actions are the set of all $\varepsilon _i^m$-mixed actions of player i in G, and player $-i$’s actions are all his mixed actions in G. By Glicksberg [28], each such game has a Nash equilibrium $\alpha ^m$. By Bolzano–Weierstrass, $\{\alpha ^m\}_{m\ge 0}$ has a convergent subsequence $\{\alpha ^k\}_{k\ge 0}$, which converges to some $\alpha$. It is straightforward to verify that $\alpha$ is a one-sided THPE for player i, with corresponding sequence $\{\alpha _i^k\}_{k\ge 0}$ in which each $\alpha _i^k$ is $\varepsilon ^k$-mixed. $\square$

Consider a NE of the stage game, $\alpha$, where player i plays a pure action, a. Assume that at the given stage, the prior probability that i is of type L is $q=q(0)$. We now construct an auxiliary strategy profile $\alpha ^q(\varepsilon )$ for any $0<\varepsilon \le q$. For player $-i$, let $\alpha ^q(\varepsilon )_{-i}(S)= \alpha ^q(\varepsilon )_{-i}(L) = \alpha _{-i}$. That is, both types of player $-i$ play their equilibrium action $\alpha _{-i}$. On the other hand, type S of player i plays the pure action a ($\alpha ^q(\varepsilon )_{i}(S,a)= 1$), and type L mixes as follows: he plays all actions other than a with equal probabilities, and action a with some probability, such that the prior probability (not knowing the type) that a is not played equals $\varepsilon$. This is possible because $0<\varepsilon \le q$.

Assume we play this stage game action profile once. If i plays an action other than a, then this reveals that the type of i is L. Otherwise, if a is played, then the probability that the player is of type L decreases. Denote this posterior probability by q(1). If it is still greater than or equal to $\varepsilon$, then the strategy $\alpha ^{q(1)}(\varepsilon )$ is well defined.

We can repeat this iteratively until one of the following occurs: Either, at some stage, the action a is not played, in which case player $-i$ learns that i is of type L. Otherwise, action a is repeatedly played until, at some stage k, the posterior probability that player i is of type L, denoted q(k), is eventually less than $\varepsilon$.

Lemma 11

There is a sequence $\{\varepsilon ^k\}_{k\ge 0}$ converging to 0 such that the following holds for each k: If we start with the prior on L equal to $q(0)=1-\beta _i$, players play the mixed actions $\alpha ^{q(\cdot )}(\varepsilon ^k)$, and a is not played in one of the first $k-1$ stages, then at the beginning of the k’th stage, the probability that player i is of type L is exactly $\varepsilon ^k$.

Proof

By construction, for each $\varepsilon$ there is some $k(\varepsilon )$ such that if we start with prior $q(0)=1-\beta _i$, players play the mixed actions $\alpha ^{q(\cdot )}(\varepsilon )$, and a is not played in one of the first $k(\varepsilon )-1$ stages, then at the $k(\varepsilon )^{th}$ stage the posterior of player i being of type L is $q(k(\varepsilon )-1)\le \varepsilon$. Denote by $g(k, \varepsilon )$ the probability that player i is an L type at the beginning of round k, given that players play the mixed actions $\alpha ^{q(\cdot )}(\varepsilon )$. Observe that for any $\varepsilon '<\varepsilon$, if $k(\varepsilon ')=k(\varepsilon )$, then $g(k(\varepsilon '),\varepsilon ')>g(k(\varepsilon ),\varepsilon )$. Furthermore, $g(k(\varepsilon ),\varepsilon )$ changes continuously with $\varepsilon$, conditional on $k(\varepsilon )$ remaining fixed.

We now iteratively construct the sequence $\{\varepsilon ^k\}_{k\ge 0}$. First, fix $\varepsilon ^0=1-\beta _i$ and the corresponding $k=0$. Next, suppose we have constructed the sequences for all $k\le {{\overline{k}}}$, and consider the case $k={{\overline{k}}}+1$. If we decrease $\varepsilon ^{{{\overline{k}}}}$ to some $\varepsilon <\varepsilon ^{{{\overline{k}}}}$, then in round ${{\overline{k}}}$ we will have $g({{\overline{k}}},\varepsilon )>g({{\overline{k}}},\varepsilon ^{{{\overline{k}}}})$. In particular $g({{\overline{k}}},\varepsilon )>\varepsilon ^{{{\overline{k}}}}$, since $g({{\overline{k}}},\varepsilon ^{{{\overline{k}}}})\ge \varepsilon ^{{{\overline{k}}}}$, and also $g({{\overline{k}}},\varepsilon )>\varepsilon$ (since $\varepsilon <\varepsilon ^{{{\overline{k}}}}$). Thus, we can add another round of the stage game. As we consider smaller and smaller $\varepsilon$, the posterior on L in round $k={{\overline{k}}}+1$ will be higher and higher. Since both $\varepsilon$ and $g({{\overline{k}}}+1,\varepsilon )$ change continuously, at some $\varepsilon$ they will be equal. Set $\varepsilon ^{{{\overline{k}}}+1}=\varepsilon$. $\square$

When a mixed action is $\varepsilon$-mixed, it may be the case that the mixing player plays all actions with probability strictly greater than $\varepsilon$. The following lemma shows that when $\varepsilon$ is small enough this is no longer the case.

Lemma 12

Suppose G is such that in every NE, player i plays a pure strategy. For any one-sided THPE $\alpha$ for player i with corresponding sequence $\{\alpha _i^k\}_{k\ge 0}$ that is $\varepsilon ^k$-mixed for each k, the mixed action $\alpha _i^k$ places weight exactly $\varepsilon ^k$ on all but at most one action.

Proof

Suppose toward a contradiction that for some k, the mixed action $\alpha _i^k$ places weight greater than $\varepsilon ^k$ on more than one action, say actions a (the pure equilibrium action) and b. Recall from the proof of Lemma 10 that $\alpha _i^k$ is a best response to $\alpha _{-i}$ out of the set of all $\varepsilon ^k$-mixed actions. But since both a and b are played with probability greater than $\varepsilon ^k$, it holds that both a and b are best responses of player i to $\alpha _{-i}$.

Now, since $\alpha$ is a NE, $\alpha _{-i}$ is a best response of player $-i$ to the pure action a of player i. By genericity assumption 1 on G, $\alpha _{-i}$ must also be a pure action, as there are no distinct pure action c and $c'$ for which $u_{-i}(a,c)=u_{-i}(a,c')$. Thus, as $\alpha _i^k$ is a best response to $\alpha _{-i}$, it is a best response to a pure action, say action c. Again by the genericity assumption, it cannot be the case that $u_i(a,c)=u_i(b,c)$. Thus, it is impossible for both a and b to be best responses to $\alpha _{-i}$, a contradiction. $\square$

Combining Lemmas 10, 11, and 12 yields the following. There exists a k with the following properties:

• 1. $\alpha _i^k$ places weight exactly $\varepsilon ^k$ on all actions other than a, and the action a is a best response of player i to $\alpha _{-i}$.

• 2. $\alpha _{-i}$ is a best response of player $-i$ to $\alpha _i^k$.

• 3. If players play the mixed actions $\alpha ^{q(\cdot )}(\varepsilon ^k)$, and a is not played in one of the first $k-1$ stages, then at the beginning of the k’th stage, the probability that player i is of type L is exactly $\varepsilon ^k$. In the k’th stage, the L type does not play action a, and so his type will be revealed with certainty.

This is thus a type-revelation phase that lasts at most k rounds. Note that in each stage game of the phase both types of player $-i$ best respond to player i, and the S type of player i best responds to player $-i$. Finally, after at most k rounds, the type of player i is revealed with certainty.