Dynamic Exploitation of Myopic Best Response

Abstract

How can a rational player manipulate a myopic best response player in a repeated two-player game? We show that in games with strategic substitutes or strategic complements the optimal control strategy is monotone in the initial action of the opponent, in time periods, and in the discount rate. As an interesting example outside this class of games we present a repeated “textbook-like” Cournot duopoly with nonnegative prices and show that the optimal control strategy involves a cycle.

A Proofs and Auxiliary Results

Proof of Lemma 1

If p is upper semicontinuous in y on Y, then by the Weierstrass Theorem an argmax exist. By Theorem of the Maximum [6], the argmax correspondence is upper hemicontinuous and compact-valued in x. Since p is strictly quasiconcave, the argmax is unique. Hence the upper hemicontinuous best-response correspondence is a continuous best-response function. Since m is upper semicontinuous and b is continuous, we have that ${\hat{m}}$ is upper semicontinuous. $\square$

Proof of Lemma 2

Under the conditions of the Lemma we have by Lemma 1 that ${\hat{m}}$ is upper semicontinuous on $X \times X$. By the Theorem of the Maximum [6], $M_1$ is upper semicontinuous on X. If $M_{n-1}$ is upper semicontinuous on X and ${\hat{m}}$ is upper semicontinuous on $X \times X$, then since $\delta \ge 0$, ${\hat{m}}(x', x) + \delta M_{n-1}(x')$ is upper semicontinuous in $x'$ on X. Again, by the Theorem of the Maximum, $M_n$ is upper semicontinuous on X. Thus by induction $M_n$ is upper semicontinuous on X for any n.

Let L be an operator on the space of bounded upper semicontinuous functions on X defined by $L M_{\infty } (x) = sup_{x' \in X_{x}} \{{\hat{m}}(x', x) + \delta M_{\infty }(x') \}$. This function is upper semicontinuous by the Theorem of the Maximum. Hence L maps bounded upper semicontinuous functions to bounded upper semicontinuous functions. L is a contraction mapping by Blackwell’s sufficiency conditions [42]. Since the space of bounded upper semicontinuous functions is a complete subset of the complete metric space of bounded functions with the $\sup$ distance, it follows from the Contraction Mapping Theorem that L has a unique fixed point $M_{\infty }$ which is upper semicontinuous on X. $\square$

Proof of Lemma 3

We state the proof just for one case. The proof of the other cases follow analogously.

• (i) If p has strongly decreasing differences in (y, x) on $Y \times X$, then by Topkis [45] b is strictly decreasing in x on X. If m has strongly decreasing differences in (x, y) on $X \times Y$, ${\hat{m}}(\cdot , \cdot ) = m(\cdot , b(\cdot ))$ must have strongly increasing differences on $X \times X$.

• (ii) If p has decreasing differences in (y, x) on $Y \times X$, then by Topkis [45] b is decreasing in x on X. Hence, if m has negative externalities, ${\hat{m}}(x', x) = m(x', b(x))$ must be increasing in x.

Proof of Proposition 1 (ii)

The proofs of the first two lines in table of Proposition 1 (ii) follow directly from previous Lemmata and Amir ([1], Theorem 2 (i)). The last two lines require a proof.

Line 3 (resp. Line 4): If m has positive externalities, and both m and p have decreasing differences (resp. m has negative externalities, and both m and p have increasing differences), and $X_y$ is contracting, then ${\bar{s}}_{n+1} \le {\bar{s}}_n$ and $\underline{s}_{n+1} \le \underline{s}_n$.

We first show that in this case $M_n(x)$ has decreasing differences in (n, x) on $\mathbb {N} \times X$. We proceed by induction by showing that for $x'' \ge x'$ and for all $n \in \mathbb {N}$,

$$\begin{aligned} M_n(x'') - M_n(x') \le M_{n-1}(x'') - M_{n-1}(x'). \end{aligned}$$

(6)

For $n = 1$, inequality (6) reduces to $M_1(x'') \le M_1(x')$ since $M_0 \equiv 0$. Since m has positive externalities and p has decreasing differences (resp. m has negative externalities and p has increasing differences), and $X_y$ is contracting, we have by Lemma 4, line 3 (resp. line 4), that $M_n$ is decreasing on X. Hence, the claim follows for $n = 1$.

Next, suppose that inequality (6) holds for all $n \in \{1, 2, \ldots , k - 1\}$. We have to show that it holds for $k = n$. Consider the maximand in Eq. (4), i.e.,

$$\begin{aligned} {\hat{m}}(z, x) + \delta M_{k - 1}(z). \end{aligned}$$

Since both m and p have decreasing differences (resp. both m and p have increasing differences), we have by Lemma 3 (i), line 2 (resp. line 1), that ${\hat{m}}(z, x)$ has increasing differences in (z, x). $M_n(z)$ has decreasing differences in (n, z) on $\{1, 2, \ldots , k - 1\} \times X$ by the induction hypothesis. Hence $M_n(z)$ has increasing differences in $(-n, x)$ on $\{-(k - 1), \ldots , -2, -1\} \times X$. We conclude that the maximand is supermodular in $(z, x, -n)$ on $X_y \times X \times \{-(k - 1), \ldots , -2, -1\}$.¹⁶ By Topkis’s ([45], Theorem 2.7.6), $M_n(x)$ has increasing differences in $(x, -n)$ on $X \times \{-k, -(k - 1), \ldots , -2, -1\}$. Thus it has decreasing differences in (x, n) on $X \times \{1, 2, \ldots , k\}$. This proves the claim that $M_n(x)$ has decreasing differences in (n, x) on $\mathbb {N} \times X$.

Finally, the dual result for decreasing differences to Topkis ([45], Theorem 2.8.3 (a)) implies that both ${\bar{s}}_{n+1} \le {\bar{s}}_n$ and $\underline{s}_{n+1} \le \underline{s}_n$. This completes the proof of line 3 (resp. line 4) in Proposition 1 (ii). $\square$

Auxiliary Result to Proposition 1(ii) Proposition 1 (ii) makes no mentioning of four other cases in which the monotone differences of m and p may differ. The following proposition show that analogous results for those cases cannot be obtained.

Proposition 3

• (i) If [m has positive externalities and decreasing differences, and p has increasing differences] or [m has negative externalities and increasing differences, and p has decreasing differences], and $X_y$ is expanding, then $M_n(x)$ has neither increasing nor decreasing differences in (n, x) unless it is a valuation.

• (ii) If [m has positive externalities and increasing differences, and p has decreasing differences] or [m has negative externalities and decreasing differences, and p has increasing differences], and $X_y$ is expanding, then $M_n(x)$ has neither increasing nor decreasing differences in (n, x) unless it is a valuation.

Proof

We just prove here part (i). Part (ii) follows analogously.

Suppose to the contrary that $M_n(x)$ has decreasing differences in (n, x). We want to show inductively that for $x'' \ge x'$ we have for all $n \in \mathbb {N}$ inequality (6). For $n = 1$, inequality (6) reduces to $M_1(x'') \le M_1(x')$ since $M_0 \equiv 0$. Since either [m has positive externalities and p has increasing differences] or [m has negative externalities and p has decreasing differences], and $X_y$ is expanding, we have by Lemma 4, line 3 (resp. line 4), that $M_n$ is increasing on X. Hence, a contradiction unless $M_1(x'') = M_1(x')$.

Suppose now to the contrary that $M_n(x)$ has increasing differences in (n, x). We want to show inductively that for $x'' \ge x'$ we have for all $n \in \mathbb {N}$,

$$\begin{aligned} M_n(x'') - M_n(x') \ge M_{n-1}(x'') - M_{n-1}(x'). \end{aligned}$$

(7)

For $n = 1$, inequality (7) reduces to $M_1(x'') \ge M_1(x')$ since $M_0 \equiv 0$. Since either [m has positive externalities and p has increasing differences] or [m has negative externalities and p has decreasing differences], and $X_y$ is expanding, we have by Lemma 4, line 3 (resp. line 4), that $M_n$ is increasing on X, which implies $M_1(x'') \ge M_1(x')$.

Furthermore, suppose that inequality (7) holds for all $n \in \{1, 2, \ldots , k-1\}$. We have to show that it holds for $k = n$. Consider the maximand in Eq. (4), i.e. ${\hat{m}}(z, x) + \delta M_{k - 1}(z)$. Since [m has decreasing differences and p has increasing differences] or [m has increasing differences and p has decreasing differences], we have by Lemma 3 (i), line 3 or 4, that ${\hat{m}}(z, x)$ has decreasing differences in (z, x). Hence ${\hat{m}}(z, x)$ has increasing differences in $(z, -x)$. $M_n(z)$ has increasing differences in (n, z) on $\{1, 2, \ldots , k - 1\} \times X$ by the induction hypothesis. We conclude that the maximand is supermodular in $(z, -x, n)$ on $X_y \times X \times \{1, 2, \ldots , k-1\}$. By Topkis’s ([45], Theorem 2.7.6), $M_n(x)$ has increasing differences in $(-x, n)$ on $X \times \{1, 2, \ldots , k-1\}$. Thus it has decreasing differences in (x, n) on $X \times \{1, 2, \ldots , k\}$, a contradiction unless it is a valuation. $\square$

Proof of Proposition 1 (iii)

The proof is essentially analogous to the proof of Theorem 2 (ii) in Amir [1]. We explicitly state where we require that ${\hat{m}}$ is increasing on X and $X_y$ is expanding.

We show by induction on n that $M_n(x, \delta )$ has increasing differences in $(x, \delta ) \in X \times (0, 1)$. For $n = 1$, the claim holds trivially since $M_1$ is independent of $\delta$.

Assume that $M_{k - 1}(x, \delta )$ has increasing differences in $(x, \delta )$. We need to show that $M_k(x, \delta )$ has increasing differences in $(x, \delta )$ has well. We rewrite Eq. (4) with explicit dependence on $\delta$ and $n = k$,

$$\begin{aligned} M_k(x, \delta ) = \max _{z \in X_y} \{{\hat{m}}(z, x) - \delta M_{k - 1}(z, \delta )\}. \end{aligned}$$

(8)

Since [both m and p have increasing differences] or [both m and p have decreasing difference], we have by Lemma 3 (i), line 1 or 2, that ${\hat{m}}(z, x)$ has increasing differences in (z, x). $M_{k - 1}(z, \delta )$ has increasing differences in $(\delta , z)$ by the induction hypothesis. That is, for $\delta '' \ge \delta '$ and $z'' \ge z'$,

$$\begin{aligned} M_{k-1}(z'', \delta '') - M_{k-1}(z', \delta '') \ge M_{k-1}(z'', \delta ') - M_{k-1}(z', \delta '). \end{aligned}$$

(9)

Since [m has positive externalities and p has increasing differences] or [m has negative externalities and p has decreasing differences] and $X_y$ is expanding, we have by Lemma 4, line 1 or 2, that $M_{k-1}(z, \delta )$ is increasing in z on $X_y$. Hence both the LHS and the RHS of inequality (9) are positive. Therefore, multiplying the LHS with $\delta ''$ and the RHS with $\delta '$ preserves the inequality. We conclude that $\delta M_{k - 1}(z, \delta )$ has increasing differences in $(\delta , z)$. Hence the maximand in Eq. (8) is supermodular in $(\delta , z, x)$ on $(0, 1) \times X_y \times X$.

By Topkis’s ([45], Theorem 2.7.6), $M_n(x, \delta )$ has increasing differences in $(\delta , x)$ on $X \times (0, 1)$. Finally, Topkis ([45], Theorem 2.8.3 (a)) implies that ${\bar{s}}_n(\cdot , \delta '') \ge {\bar{s}}_n(\cdot , \delta ')$ and $\underline{s}_n(\cdot , \delta '') \ge \underline{s}_n(\cdot , \delta ')$. This completes the proof of Proposition 1 (iii). $\square$

Auxiliary Results to Proposition 1(iii) Proposition 1 (ii) is silent on a number of cases:

Proposition 4

Suppose that [m has positive externalities, m has decreasing differences, and p has increasing differences] or [m has negative externalities, m has increasing differences, and p has decreasing differences] and $X_y$ is expanding. Then $M_n(x, \delta )$ has NOT increasing differences in $(\delta , x)$ on $(0, 1) \times X$ unless it is a valuation.

Proof

Suppose to the contrary that $M_n(x, \delta )$ has increasing differences in $(\delta , x) \in (0, 1) \times X$. For $n = 1$ the claim is trivial since $M_n$ is independent of $\delta$.

Assume that $M_{k - 1}(x, \delta )$ has increasing differences in $(x, \delta )$. We need to show that $M_k(x, \delta )$ has increasing differences in $(x, \delta )$ has well. Consider the maximand in Eq. (8). Since [m has decreasing differences and p has increasing differences] or [m has increasing differences and p has decreasing difference], we have by Lemma 3 (i), line 3 or 4, that ${\hat{m}}(z, x)$ has decreasing differences in (z, x). Hence, it has increasing differences in $(z, -x)$. $M_{k - 1}(z, \delta )$ has increasing differences in $(\delta , z)$ by the induction hypothesis so that inequality (9) holds.

Since [m has positive externalities and p has increasing differences] or [m has negative externalities and p has decreasing differences] and $X_y$ is expanding, we have by Lemma 4, line 1 or 2, that $M_{k-1}(z, \delta )$ is increasing in z on $X_y$. Hence both the LHS and the RHS of inequality (9) are positive. Therefore, multiplying the LHS with $\delta ''$ and the RHS with $\delta '$ preserves the inequality. We conclude that $\delta M_{k - 1}(z, \delta )$ has increasing differences in $(\delta , z)$. Hence the maximand in Eq. (8) is supermodular in $(\delta , z, -x)$ on $(0, 1) \times X_y \times X$.

By Topkis’s ([45], Theorem 2.7.6), $M_n(x, \delta )$ has increasing differences in $(\delta , -x)$ on $X \times (0, 1)$. Hence it has decreasing differences in $(\delta , x)$, a contradiction unless it is a valuation. $\square$

Two other cases, namely

• (i) [m has positive externalities and both m and p have decreasing differences] or [m has negative externalities and both m and p have increasing differences] and $X_y$ is contracting,

• (ii) [m has positive externalities, increasing differences, and p has decreasing differences] or [m has negative externalities, decreasing differences, and p has increasing differences] and $X_y$ is contracting,

cannot be dealt with the method used to prove Proposition 1 (iii) and Proposition 4. Both cases are such that according to Lemma 4 we have that $M_n(x, \delta )$ is decreasing on X. Therefore the analogous inequality to (9) may be reversed if multiplying the LHS with $\delta ''$ and the RHS with $\delta '$.

Proof of Proposition 2 (i)

Note that the first-order condition for the maximization in Eq. (5) (analogously for Eq. (4)) is

$$\begin{aligned} \frac{\partial {\hat{m}}(x, s(x))}{\partial z} + \delta \frac{\partial M(s(x))}{\partial x} = 0. \end{aligned}$$

(10)

Suppose that for some $x'' > x'$, $s(x'') = s(x')$. Then from Eq. (10) we conclude $\frac{\partial {\hat{m}}(x'', s(x''))}{\partial z} = \frac{\partial {\hat{m}}(x', s(x'))}{\partial z}$, which contradicts that ${\hat{m}}$ has strongly decreasing differences in (x, z). Hence, $s(x'') = s(x')$ is not possible, and then by Proposition 1 (i), $s(x'') < s(x')$. This completes the proof of part (i). $\square$

Proof of Proposition 2 (ii)

The proof is essentially “dual” to the proof of Amir ([1], Theorem 3 (ii)).

By Proposition 1 (ii) that $s_{n + 1}(x) \ge s_n(x)$ for all $x \in X$. Suppose that for some $x_n \in X$, $s_{n + 1}(x_n) = s_{n}(x_n)$. We will show that there exists $x' \in X$ such that $s_{n-1}(x') = s_{n-2}(x')$.

Plugging $s_{n + 1}(x_n) = s_{n}(x_n)$ in the Euler equations corresponding to the problem given in Eq. (4) for $n = 2, 3, \ldots$

$$\begin{aligned}&\frac{\partial {\hat{m}}(s_n(x_n), x_n)}{\partial z} + \delta \frac{\partial {\hat{m}}(s_{n-1}(s_n(x_n)), s_n(x_n))}{\partial x} = 0, \end{aligned}$$

(11)

$$\begin{aligned}&\frac{\partial {\hat{m}}(s_{n+1}(x_n), x_n)}{\partial z} + \delta \frac{\partial {\hat{m}}(s_{n}(s_{n+1}(x_n)), s_{n+1}(x_n))}{\partial x} = 0, \end{aligned}$$

(12)

leads to

$$\begin{aligned} \frac{\partial {\hat{m}}(s_{n-1}(s_n(x_n)), s_n(x_n))}{\partial x} = \frac{\partial {\hat{m}}(s_{n}(s_{n+1}(x_n)), s_{n+1}(x_n))}{\partial x}. \end{aligned}$$

Since ${\hat{m}}$ has strongly increasing differences by Lemma 3 (i) we must have $s_{n-1}(s_n(x_n)) = s_{n}(s_{n+1}(x_n))$. Hence $s_{n-1}(s_n(x_n)) = s_n(s_{n}(x_n))$. Set $x_{n-1} \equiv s_n(x_n)$. Thus $s_{n-1}(x_{n-1}) = s_n(x_{n-1})$. Plugging into the Euler equations,

$$\begin{aligned}&\frac{\partial {\hat{m}}(s_{n-1}(x_{n-1}), x_{n-1})}{\partial z} + \delta \frac{\partial {\hat{m}}(s_{n-2}(s_{n-1}(x_{n-1})), s_{n-1}(x_{n-1}))}{\partial x} = 0, \end{aligned}$$

(13)

$$\begin{aligned}&\frac{\partial {\hat{m}}(s_{n}(x_{n-1}), x_{n-1})}{\partial z} + \delta \frac{\partial {\hat{m}}(s_{n-1}(s_{n}(x_{n-1})), s_{n}(x_{n-1}))}{\partial x} = 0, \end{aligned}$$

(14)

leads to

$$\begin{aligned} \frac{\partial {\hat{m}}(s_{n-2}(s_{n-1}(x_{n-1})), s_{n-1}(x_{n-1}))}{\partial x} = \frac{\partial {\hat{m}}(s_{n-1}(s_{n}(x_{n-1})), s_{n}(x_{n-1}))}{\partial x}. \end{aligned}$$

Since ${\hat{m}}$ has strongly increasing differences by Lemma 3 (i) last equation implies that $s_{n-1}(s_{n}(x_{n-1})) = s_{n-2}(s_{n-1}(x_{n-1})) = s_{n-2}(s_n(x_{n-1}))$. Hence there exists $x' \in X$ such that $s_{n-1}(x') = s_{n-2}(x')$.

By induction we obtain the existence of $x_2 \in X$ for which $s_1(x_2) = s_2(x_2)$. The Euler equations for the one- and two-period problems at $x_2$ are given by

$$\begin{aligned}&\frac{\partial {\hat{m}}(s_1(x_2), x_2)}{\partial z} = 0, \end{aligned}$$

(15)

$$\begin{aligned}&\frac{\partial {\hat{m}}(s_2(x_2), x_2)}{\partial z} + \delta \frac{\partial {\hat{m}}(s_1(s_2(x_2)), s_2(x_2))}{\partial x} = 0. \end{aligned}$$

(16)

Since $x_2 \in X$ is such that $s_1(x_2) = s_2(x_2)$, the Euler equations imply $\frac{\partial {\hat{m}}(s_1(s_2(x_2)), s_2(x_2))}{\partial x} = 0$.

Note that the conditions of line 3 or 4 in Proposition 2 (ii) imply by Lemma 3 (ii) that $\frac{\partial {\hat{m}}(z, x)}{\partial x} < 0$, a contradiction. $\square$