Critical Discount Factor Values in Discounted Supergames

Abstract

This paper examines the subgame-perfect equilibria in symmetric $2\times 2$ supergames. We solve the smallest discount factor value for which the players obtain all the feasible and individually rational payoffs as equilibrium payoffs. We show that the critical discount factor values are not that high in many games and they generally depend on how large the payoff set is compared to the set of feasible payoffs. We analyze how the stage-game payoffs affect the required level of patience and organize the games into groups based on similar behavior. We study how the different strategies affect the set of equilibria by comparing pure, mixed and correlated strategies. This helps us understand better how discounting affects the set of equilibria and we can identify the games where extreme patience is required and the type of payoffs that are difficult to obtain. We also observe discontinuities in the critical values, which means that small changes in the stage-game payoffs may affect dramatically the equilibrium payoffs.

1. Introduction

The folk theorem tells us that any feasible and individually rational (FIR) payoff is an equilibrium, when the players are patient enough [1,2,3,4]. However, the players may not be extremely patient but they rather have some intermediate value for the discount factor. This paper finds the smallest discount factor value for which the FIR payoffs are the equilibrium payoffs in the symmetric $2\times 2$ supergames. This extends the folk theorem by solving exactly how patient the players need to be as a function of the stage-game payoffs. This reveals why and in what type of games extreme patience is required in the folk theorem and what payoffs are difficult to obtain.

This paper compares three types of strategies: pure strategies with and without public correlation and mixed strategies without public correlation. We want to examine how these assumptions affect the results. In some applications, it may be reasonable to assume that the players may only use pure strategies and they may not be able to coordinate their actions using a correlation device. The pure-strategy model has been examined in [5,6,7,8]. These papers characterize the subgame-perfect equilibria with a set-valued fixed-point equation, which forms the basis of our analysis. The mixed-strategy model has been studied in the more general model of imperfect public monitoring [9,10,11]; see also [12,13] for the model examined in this paper.

The computation of the payoff set has been examined in [14,15,16,17,18]. These papers assume public correlation, which makes the payoff set convex and simplifies the computation dramatically. References [19,20] have developed a method for computing pure-strategy equilibria without public correlation.

The main result of this paper is to solve the critical values, i.e., the smallest discount factor value for which the players obtain all the FIR payoffs as equilibrium payoffs. The results are based on solving analytically and geometrically the fixed-point equation of [7,8]. This idea has been presented in [21] where the critical values are solved in a class of prisoner’s dilemma games under public randomization; see also Sections 2.5.3 and 2.5.6 in [11]. The pure-strategy model without public correlation is a quite straightforward extension of [21], but the mixed-strategy model requires analyzing totally new types of strategies with complicated payoffs; see [12,13]. We restrict our analysis to the symmetric $2\times 2$ games since the set of pure-strategy equilibria may be empty in asymmetric games, it may be difficult to find the smallest equilibrium payoffs [22], and it is tedious to examine all the asymmetric games.

The results of this paper can be used (i) to identify the games where the players have trouble in obtaining all the payoffs and to find the payoffs that are difficult to obtain, (ii) to understand better how the equilibrium payoffs depend on the discount factor, and (iii) to find a range of discount factor values for which the computation of equilibria is easy. For example, we know that the payoff set coincides with the FIR payoffs for all the discount factor values above the critical value, and there is no need to compute the payoff set for these values. We also observe discontinuities in the critical values, which means that the set of equilibria may not behave well, i.e., small changes in the stage-game payoffs may affect dramatically the payoff set.

We organize the $2\times 2$ games into groups based on the equations that determine the critical values and provide a useful visualization of the critical values in different games. From the figures, it is easy to see when the high level of patience is needed and to make the comparison between the different strategies.

2. The Repeated Game

2.1. Stage Games

In a repeated game, a stage game is played again and again by the same players. A stage game is defined by a finite set of players $N=\{1,\dots ,n\}$, a finite set of pure actions for each player $A_i$, $i\in N$, and the players’ utilities for each action profile $u:A\mapsto {\mathbb{R}}^n$, where $A={\times }_{i\in N}{A}_i$ is the set of pure-action profiles. Moreover, a pure action of player i is denoted $a_i\in A_i$ and a pure-action profile is $a\in A$.

Each player $i\in N$ may randomize over his pure actions $a_i\in A_i$. This defines a mixed action $q_i$ such that $q_i\left(a_i\right)\ge 0$ for each $a_i\in A_i$ and ${\sum }_{a_i\in A_i}{q}_i\left(a_i\right)=1$. The set of probability distributions over $A_i$ is denoted $Q_i$ and $Q={\times }_{i\in N}{Q}_i$. A mixed-action profile is denoted by $q=(q_1,\dots ,q_n)\in Q$. A support of a mixed action is the set of pure actions that is played with a strictly positive probability: $Supp\left(q_i\right)=\{a_i\in A_i|q_i\left(a_i\right)>0\}$. We also define $Supp\left(q\right)={\times }_{i\in N}Supp\left(q_i\right)$, and for each $a\in Supp\left(q\right)$, we let ${\pi }_q\left(a\right)$ be the probability that the action profile a is realized if the mixed-action profile q is played: ${\pi }_q\left(a\right)={\prod }_{j\in N}{q}_j\left(a_j\right)$. In pure strategies, we make the restriction that $q_i\left(a_i\right)=1$ for one action $a_i\in A_i$.

In a model with public correlation, the players observe a realization $\omega \in [0,1]$ of a public lottery and they can condition their action based on the signal $\omega$. For example, two players may agree to take action $a_1$ if $\omega \le 1/2$ and $a_2$ otherwise. This way the players can coordinate their actions such that they randomize between the outcomes $(a_1,a_1)$ and $(a_2,a_2)$, and avoid the outcomes $(a_1,a_2)$ and $(a_2,a_1)$. The latter outcomes would be realized if standard mixed strategies were used and no public correlation device was available.

The stage-game payoffs are given by function $u:Q\mapsto {\mathbb{R}}^n$. For example, if the players choose a mixed-action profile $q\in Q$, then player i receives an expected payoff of

$$u_i\left(q\right)=\sum _{a\in A}{u}_i\left(a\right){\pi }_q\left(a\right).$$

(1)

Let $q_{-i}\in Q_{-i}={\times }_{j\in N,j\ne i}{Q}_j$ denote player i’s opponents’ actions. Now, an action profile q is a Nash equilibrium in the stage game if no player has a profitable deviation, i.e.,

$$u_i\left(q\right)\ge u_i(q_i^{\prime },q_{-i})\mathrm{for} \mathrm{all}{q}_i^{\prime }\in Q_i\mathrm{and}i\in N.$$

(2)

The twelve symmetric strict ordinal $2\times 2$ games are presented in Figure 1, see ref. [23] for the taxonomy. Strict ordinality means that all of the payoffs must be unequal and there can be no indifferences. The two actions are C (cooperate) and D (defect), and they give the players the payoffs $\mathfrak{a}=1$, $\mathfrak{b}$, $\mathfrak{c}$ and $\mathfrak{d}=0$; the corresponding action profiles are also denoted by letters $a,b,c,d$. For example, if the players choose the action profile $b=(C,D)$, the players receive payoffs $\mathfrak{b}$ and $\mathfrak{c}$.

2.2. Repeated Games

We examine a model where the stage game is repeated infinitely many times and these games are sometimes called as supergames. We assume that the players observe all the past realized pure actions but not the probabilities that the other players are using in their mixed strategies. The public past play is denoted by the set of histories $H^k=A^k={\prod }_{k}A$, where $H^0=A^0=\{⌀\}$ is the empty set and corresponds to the beginning of the game. Thus, the history contains all the pure actions that were played in the previous rounds. The set of all possible histories is $H={\bigcup }_{k=0}^{\infty }{H}^k$. A behavior strategy ${\sigma }_i$ of player $i\in N$ is a mapping that assigns a probability distribution over player i’s pure actions for each possible history ${\sigma }_i:H\mapsto Q_i$. The set of player i’s strategies is ${\Sigma }_i$. The players’ strategies form a strategy profile $\sigma =({\sigma }_1,\dots ,{\sigma }_n)$, a strategy profile of all players except player i is denoted by ${\sigma }_{-i}$ and the set of strategy profiles is given by $\Sigma ={\times }_{i\in N}{\Sigma }_i$. A pure strategy assigns a pure action for each possible history ${\sigma }_i:H\mapsto A_i$. With public correlation, the players observe a public signal ${\omega }^k\in [0,1]$ on each round k before making their decisions, and thus the history contains all the past pure actions, signals and the current signal.

We assume that the players discount the future payoffs with a common discount factor $\delta \in \left[0,1\right)$. They have the same discount factor since we examine symmetric games. The expected discounted payoff of a strategy profile $\sigma$ to player i is

$$U_i\left(\sigma \right)=\mathbb{E}\left[(1-\delta )\sum _{k=0}^{\infty }{\delta }^k{u}_i^k\left(\sigma \right)\right],$$

(3)

where $u_i^k\left(\sigma \right)$ is the payoff of player i at round k induced by the strategy profile $\sigma$. A strategy profile $\sigma$ is a Nash equilibrium if no player has a profitable deviation, i.e.,

$$U_i\left(\sigma \right)\ge U_i({\sigma }_i^{\prime },{\sigma }_{-i})\mathrm{for} \mathrm{all}{\sigma }_i^{\prime }\in {\Sigma }_i\mathrm{and}i\in N,$$

(4)

and it is a subgame-perfect equilibrium (SPE) if it induces a Nash equilibrium in every subgame, i.e.,

$$U_i\left(\sigma \right|h)\ge U_i({\sigma }_i^{\prime },{\sigma }_{-i}|h)\mathrm{for} \mathrm{all}{\sigma }_i^{\prime }\in {\Sigma }_i,h\in H,\mathrm{and}i\in N,$$

(5)

where $\sigma |h$ is the restriction of the strategy profile after history $h\in H$. From now on, by equilibrium we mean subgame-perfect equilibrium.

2.3. Critical Discount Factor Values

Let V be the compact set of SPE payoffs and we also use $V\left(\delta \right)$ when we want to emphasize the players’ discount factor $\delta$. By $V^C$, $V^P$ and $V^M$ we refer to the equilibria in pure strategies with and without public correlation, and mixed strategies, respectively.

The player i’s minimum equilibrium payoff, which is also called the punishment payoff, is denoted by $v_i^{-}\left(\delta \right)=min\{v_i:v\in V\left(\delta \right)\}$, when $V\left(\delta \right)$ is non-empty; and this is the case in the symmetric $2\times 2$ supergames. Similarly, the maximum equilibrium payoff is $v_i^{+}\left(\delta \right)=max\{v_i:v\in V\left(\delta \right)\}$. The minimax payoff is

$${\underline{v}}_i=\underset{q_{-i}\in Q_{-i}}{min}\underset{q_i\in Q_i}{max}{u}_i(q_i,q_{-i}).$$

(6)

Please note that the minimax payoffs can be different in pure and mixed strategies. However, it holds under perfect monitoring that $v_i^{-}\left(\delta \right)\ge {\underline{v}}_i$, $\forall i\in N$ [4,11]. The player’s minimum and maximum payoffs in a compact set W are denoted by $v_i^{-}\left(W\right)$ and $v_i^{+}\left(W\right)$, respectively. Moreover, $v^s\left(W\right)=max\{v_i\in W,v_i=v_j,\forall j\in N\}$ is the maximum symmetric payoff in the set W.

Let $V^{†}=co\left(v\in {\mathbb{R}}^n:\exists q\in A\mathrm{s}.\mathrm{t}.v=u\left(q\right)\right)$ be the set of feasible payoffs, where $co$ denotes the convex hull of the set. The set of feasible and individually rational (FIR) payoffs are

$$V^{\ast }=\{v\in V^{†}:v_i\ge {\underline{v}}_i,i\in N\}.$$

Let us denote the critical discount factor by

$${\delta }^F=min\{\delta :V\left(\delta \right)=V^{\ast }\},$$

(7)

which gives the smallest discount factor value when the payoff set coincides with the FIR payoffs. Please note that $V^{\ast }$ is convex and thus $V\left(\delta \right)=V^{\ast }$ for all $\delta \ge {\delta }^F$ by Theorem 3.

Please note that the minimum equilibrium payoff $v_i^{-}\left(\delta \right)$ may be strictly higher than the minimax payoff ${\underline{v}}_i$, and then it is impossible to obtain all the FIR payoffs for given $\delta <1$. The minimum pure-strategy payoffs have been studied in [24], and [22] present an algorithm for finding the punishment paths and payoffs.

For most of the symmetric $2\times 2$ games, the minimum equilibrium payoffs in pure strategies are equal to the minimax payoffs for all discount factors. No conflict, its anti-game and anti-stag hunt games are the only exceptions. In these games, the minimum payoffs are equal to the minimax values when the players are patient enough. This issue does not affect the results, since it can be shown the required levels of patience are smaller than the critical values.

The minimax payoffs in mixed strategies are the same as in pure strategies, except in leader, battle of the sexes, coordination and anti-coordination games. In these games, the minimax payoff is given by a mixed-strategy Nash equilibrium. Thus, $v^{-}\left(\delta \right)=\underline{v}$ for all $\delta$ in mixed strategies. It can be shown that the FIR payoffs are not obtained for any $\delta <1$ in these games.

2.4. The Characterization of Equilibria

A pair $(a,w)$ of an action profile $a\in A$ and a continuation payoff $w\in W$ is admissible with respect to W if it satisfies

$$(1-\delta )u_i\left(a\right)+\delta w_i\ge \underset{a_i^{\prime }\in A_i}{max}\left[(1-\delta )u_i\left(a_i^{\prime },a_{-i}\right)+\delta v_i^{-}\left(W\right)\right],\forall i\in N.$$

(8)

This incentive-compatibility condition means that it is better for player i to take action $a_i$ and get the continuation payoff $w_i$ than to deviate and then obtain $v_i^{-}\left(W\right)$.

For a set of continuation payoffs W, the set of supportable action profiles is denoted by

$$F^{\delta }\left(W\right)=\{a\in A\mathrm{such}\mathrm{that}(a,w)\mathrm{is}\mathrm{admissible}\mathrm{for}\mathrm{some}w\in W\}.$$

(9)

For $a\in F^{\delta }\left(W\right)$, we denote the set of admissible continuation payoffs as

$$C_a^{\delta }\left(W\right)=\{w\in W\mathrm{such}\mathrm{that}(a,w)\mathrm{is}\mathrm{admissible}\}.$$

(10)

Let $D_a^{\delta }:{\mathbb{R}}^n\mapsto {\mathbb{R}}^n$ be an affine mapping that corresponds to an action profile $a\in A$ and a discount factor $\delta$

$$D_a^{\delta }\left(w\right)=(I-T)u\left(a\right)+Tw,$$

(11)

where I is an $n\times n$ identity matrix and T is an $n\times n$ diagonal matrix with the discount factor $\delta$ on the diagonal. The mapping $D_a^{\delta }$ is also defined for sets; then the addition is the Minkowski sum and $D_a^{\delta }(⌀)=⌀$. Finally, we denote the admissible payoffs that start with an action profile $a\in A$ by $B_a^{\delta }\left(W\right)=D_a^{\delta }\left(C_a^{\delta }\left(W\right)\right)$.

Theorem 1.

The set of pure-strategy subgame-perfect equilibrium payoffs $V^P$ is the unique largest compact set satisfying the fixed-point [8,11]

$$W=B^{\delta }\left(W\right)\doteq \bigcup _{a\in F^{\delta }\left(W\right)}{B}_a^{\delta }\left(W\right).$$

(12)

The payoff set under public correlation $V^C$ is given by the largest compact set satisfying [11]

$$W={\mathfrak{B}}^{\delta }\left(W\right)\doteq co\left(\bigcup _{a\in F^{\delta }\left(V\right)}{B}_a^{\delta }\left(W\right)\right).$$

(13)

Let us now characterize the set of equilibria in mixed strategies [12,13]. In a repeated game, the play at each round is strategically equivalent to playing an augmented stage game, where the continuation payoffs are included in the payoffs. For each action profile $a\in A$, the payoff in the augmented game is given by

$${\tilde{u}}_{\delta }\left(a\right)\doteq (1-\delta )u\left(a\right)+\delta x\left(a\right),$$

where $x\left(a\right)$ is the continuation payoff after a. Please note that in pure strategies there are only two continuation payoffs for each player: $w_i$ if the player follows the equilibrium path or $v_i^{-}$ if the player deviates. In mixed strategies, there can be a different continuation payoff $x\left(a\right)$ after each action profile $a\in A$. Let $M\left(u\right(a\left)\right)$ denote the set of Nash equilibrium payoffs in a stage game with payoffs $u\left(a\right)$, $a\in A$.

Now, we are ready to state the characterization for the subgame-perfect equilibrium payoffs [12,13]. This result has been derived earlier in a more general model of imperfect monitoring [11].

Theorem 2.

The payoff set $V^M$ is the largest compact fixed point of $\mathcal{B}$:

$$W=\mathcal{B}\left(W\right)\doteq \bigcup _{x\left(a\right)\in W}M((1-\delta )u\left(a\right)+\delta x\left(a\right)).$$

(14)

This means that the payoff set $V^M$ corresponds to the set of equilibria in augmented stage games where the payoffs are $(1-\delta )u\left(a\right)+\delta x\left(a\right)$ and each $x\left(a\right)$ can be chosen from the set $V^M$.

2.5. Monotonicity and Helpful Results

In this subsection, by $B\left(V\right)$ we refer to the sets $B\left(V\right)$, $\mathfrak{B}\left(V\right)$ and $\mathcal{B}\left(V\right)$, depending on which strategies are in question. A set W is called self-generating if $W\subseteq B^{\delta }\left(W\right)$. The following result follows directly from Theorems 1 and 2 and Equation (13).

Proposition 1.

If a bounded set W is self-generating then $B^{\delta }\left(W\right)\subseteq V\left(\delta \right)$.

The following shows that the payoff set is monotone in the discount factor when it is convex [8,11,24,25].

Theorem 3.

Suppose $V\left({\delta }_1\right)$ is convex then $V\left({\delta }_1\right)\subseteq V\left({\delta }_2\right)$ for ${\delta }_2\ge {\delta }_1$.

Let $v^A\left(W\right)$, $v^B\left(W\right)$, $v^C\left(W\right)$ and $v^D\left(W\right)$ be the corners of a quadrilateral set W corresponding to payoffs $u\left(a\right)$, $u\left(b\right)$, $u\left(c\right)$ and $u\left(d\right)$. For example, if $u\left(a\right)$ and $u\left(b\right)$ are the payoffs in the northeast and northwest corners of $V^{†}$, then $v^A\left(B_b\left(\delta \right)\right)$ and $v^B\left(B_a\left(\delta \right)\right)$ are the northeast and the northwest corners of the sets $B_b$ and $B_a$; see Figure 2a. Moreover, let

$$v^M\left(\delta \right)=(1-\delta )\underset{a\in A}{max}{u}_i\left(a\right)+\delta v_i^{-}\left(\delta \right).$$

(15)

This is the right-hand side of Equation (8) for the column that contains the maximum payoff in the game.

Remark 1.

If $B_a\left(\delta \right)\ne ⌀$, $a\in A$, then $v^M\left(\delta \right)=v_i^{-}\left(B_a\left(\delta \right)\right)$ for all a and i where player i can deviate to the maximum payoff of the stage game.

The following result describes how the sets $B_a$, $a\in A$, may cover the boundaries of $V^{\ast }$. The result implies that we need as many sets $B_a$, $a\in A$, to cover the FIR payoffs as there are corner points in $V^{\ast }$.

Proposition 2.

The set $B_a^{\delta }\left(V^{\ast }\right)$, $a\in A$, may only cover the corner point of $V^{\ast }$ closest to $u\left(a\right)$. It cannot cover the other corner points or the boundary of $V^{\ast }$ between these other corner points.

Proof. 

By the definition of $D_q^{\delta }$, the set $V^{\ast }$ is contracted by $\delta$ and is thus strictly smaller than $V^{\ast }$. The translation part $(I-T)u\left(a\right)$ moves the set towards $u\left(a\right)$. ☐

3. Results for Different Strategies

We present now the results for the three different strategies, and the proofs are given in the appendix. Section 4 gives an example of the proofs in a prisoner’s dilemma game. The proofs for the mixed strategies are novel in Appendix C. However, the principle behind the proofs is the same but finding all the mixed-strategy payoffs is more complicated.

The results are based on Theorems 1 and 2 and Equation (13), which tell that the payoff set coincides with the FIR payoffs when the admissible payoffs cover all the payoffs in $V^{\ast }$. To find the critical value ${\delta }^F$, we need to find the smallest discount factor for this to happen. The main idea is to find the last payoff point $v^F\in V^{\ast }$ that is covered when the discount factor is increased to ${\delta }^F$. The value ${\delta }^F$ is typically solved from a condition that two sets, say $B_b$ and $B_c$ intersect. In the proofs, we show both the necessary and sufficient conditions for ${\delta }^F$: the point $v^F$ is not covered for a smaller discount factor value and all the other FIR payoffs are covered for the given value ${\delta }^F$.

The results are given in

Table 1. The values of the discount factor ${\delta }^F$ without public correlation. Some games have multiple groups, and the equation that gives the boundary is shown on the right.

Game	First Group	Second Group	Group Boundary
Prisoner’s dilemma	I: ${\delta }^F=\frac{\mathfrak{c}-\mathfrak{b}}{1+\mathfrak{c}-\mathfrak{b}}$	IIIb: ${\delta }^F=\frac{2\mathfrak{c}}{\mathfrak{b}+3\mathfrak{c}}$	$\mathfrak{c}=2-\mathfrak{b}$
Chicken	IIId: ${\delta }^F=\frac{\mathfrak{c}}{1+\mathfrak{c}-\mathfrak{b}}$	IIIb: ${\delta }^F=\frac{2\mathfrak{c}}{3\mathfrak{c}-\mathfrak{b}}$
Leader	IIIb: ${\delta }^F=\frac{2\mathfrak{c}}{3\mathfrak{c}-\mathfrak{b}}$
Battle of the sexes	IIIc: ${\delta }^F=\frac{2\mathfrak{b}-2}{3\mathfrak{b}-\mathfrak{c}-2}$
Anti-PD	V: ${\delta }^F=0$	IIIc: ${\delta }^F=\frac{2\mathfrak{b}-2}{3\mathfrak{b}-\mathfrak{c}-2}$	$\mathfrak{c}=2-\mathfrak{b}$
Anti-chicken
Stag hunt	II: ${\delta }^F=\frac{1-\mathfrak{b}}{1-2\mathfrak{b}+\mathfrak{c}}$	IIIa: ${\delta }^F=\frac{3-3\mathfrak{b}-2\mathfrak{c}+\mathfrak{b}(\mathfrak{b}+\mathfrak{c})}{5-4\mathfrak{b}+{\mathfrak{b}}^2-3\mathfrak{c}+\mathfrak{bc}}$	$\mathfrak{c}=1+\left(\mathfrak{b}-1\right)\sqrt{\frac{\mathfrak{b}}{\mathfrak{b}-2}}$
Coordination
No conflict	II: ${\delta }^F=\frac{1+\mathfrak{b}(\mathfrak{c}-2)}{1+\mathfrak{b}(\mathfrak{b}-3)+\mathfrak{c}}$
Anti-coordination	IV: ${\delta }^F=1$
Anti-no conflict
Anti-stag hunt

,

Table 2. The values of discount factor ${\delta }^F$ with public correlation.

Game	First Group	Second Group	Third Group
Prisoner’s dilemma	II: ${\delta }^F=(\mathfrak{c}-1)/\mathfrak{c}$	Ib: ${\delta }^F=\mathfrak{b}/(\mathfrak{b}-1)$	Ia: ${\delta }^F=-\mathfrak{b}/\mathfrak{c}$
Stag hunt	Ia: ${\delta }^F=(1-\mathfrak{c})/(1-\mathfrak{b})$
Coordination
No conflict		IIIa: ${\delta }^F=\mathfrak{b}$
Chicken	IIIa: ${\delta }^F=2\mathfrak{b}/(\mathfrak{b}+\mathfrak{c})$		II: ${\delta }^F=\frac{\mathfrak{c}-1}{\mathfrak{c}-\mathfrak{b}}$
Leader
Anti-PD	V: ${\delta }^F=0$	Ia: ${\delta }^F=\frac{\mathfrak{c}-1}{1-\mathfrak{b}}$
Anti-chicken
Battle of the sexes	IIIb: ${\delta }^F=(2-2\mathfrak{c})/(2-\mathfrak{b}-\mathfrak{c})$
Anti-coordination	IV: ${\delta }^F=1$
Anti-no conflict
Anti-stag hunt

and

Table 3. The values of discount factor ${\delta }^F$ in mixed strategies.

Game	First Group	Second Group	Third Group
Prisoner’s dilemma	Ib: ${\delta }^F=(\mathfrak{c}-\mathfrak{b})/2\mathfrak{c}$	Ia: ${\delta }^F=(\mathfrak{c}-\mathfrak{b})/(1+\mathfrak{c}-\mathfrak{b})$
Chicken	III ${}^{\ast }$: ${\delta }^F=(\mathfrak{b}+\mathfrak{c})/2\mathfrak{c}$	Ia: ${\delta }^F=(\mathfrak{c}-\mathfrak{b})/(1+\mathfrak{c}-2\mathfrak{b})$	III ${}^{\ast }$: ${\delta }^F=\frac{1}{2-\mathfrak{b}}$
Leader ${}^{\ast \ast }$
Battle of the sexes ${}^{\ast \ast }$	III ${}^{\ast }$: ${\delta }^F=\frac{\mathfrak{b}+\mathfrak{c}-2}{2(\mathfrak{b}-1)}$
Stag hunt	IIa: ${\delta }^F=(1-\mathfrak{c})/(1-\mathfrak{b})$	Ia: ${\delta }^F=(\mathfrak{b}-1)/(\mathfrak{b}-2)$
Coordination ${}^{\ast \ast }$
No conflict		Ia: ${\delta }^F=1/2$	IIb ${}^{\ast }$: ${\delta }^F=\mathfrak{b}$
Anti-no conflict	IV: ${\delta }^F=1$
Anti-coordination ${}^{\ast \ast }$
Anti-stag hunt
Anti-chicken	V: ${\delta }^F=0$	Ib: ${\delta }^F=(\mathfrak{b}-\mathfrak{c})/2(\mathfrak{b}-1)$
Anti-PD

${}^{\ast }$ Possibly only an upper bound. ** ${\delta }^F=1$, the values correspond to obtaining the pure-strategy FIR payoffs.

. They show the values of ${\delta }^F$ for different groups of games. Please note that the group boundaries cross the game boundaries. For example, there are two groups of prisoner’s dilemma (PD) games in pure strategies without public correlation: the quadrilateral PDs belong to Group I (see example in Figure 2a) and the triangle PDs in Group IIIb, and the boundary between these groups is given by the equation $\mathfrak{c}=2-\mathfrak{b}$. Also, different games may belong to the same group: the triangle PDs and triangle chicken games all belong to Group IIIb. Figure 3 shows the groups for the different strategies. The thick grey lines show that there are discontinuities of ${\delta }^F$ between the groups. This means that the value of ${\delta }^F$ is not continuous and there may be a jump, when b and c are changed. Figure 4 shows the values of ${\delta }^F$ in different games.

The groups are based on the equations that determine the value of ${\delta }^F$ and the location of the last payoff $v^F$. We note that our classification is heuristic, and the games could be organized differently into groups. In pure strategies without public correlation: in Group I, $v^F$ is found on the upper edge of $V^{\ast }$ between $u\left(a\right)$ and $u\left(b\right)$, on the bottom edge between $u\left(b\right)$ and $u\left(d\right)$ in Group II, and in the middle in Groups IIIa-d. In Group IV, we have ${\delta }^F=1$ and in Group V, ${\delta }^F=0$.

With public correlation, the groups are based on the last corner point to be covered: in Group I, the last corner is in the northwest, corresponding to $u\left(b\right)$ or $u\left(c\right)$, the $u\left(a\right)$ corner in Group II, and the $u\left(d\right)$ corner in Group III.

In mixed strategies, the groups are the following. In Group Ia, the last point to be covered is on the boundary between $u\left(b\right)$ and $u\left(a\right)$, i.e., the intersection of $B_a$ and $B_b$. Group Ib corresponds to the triangle games, where $B_b$ and $B_c$ intersect. In Group II, a corner point determines the value of ${\delta }^F$: $u\left(b\right)$ corner is last to be covered in Group IIa and $u\left(d\right)$ corner in Group IIb. The intersection of $B_d$ and $B_b$ determines the value for Group III.

The overview of the values of ${\delta }^F$ is similar for all the strategies. Groups IV and V are the same, and the locations of the high and low values are about the same. However, the values of ${\delta }^F$ are much lower with public correlation. The scale with public correlation is between 0 and 1, and between $1/2$ or $2/3$ to 1 with the other strategies. The smallest and the highest values within the game classes are shown with $z_1$ to $z_4$. The difference between the pure and mixed strategies is surprisingly small; the values are typically smaller than $0.05$ in quadrilateral games and smaller than $0.15$ in triangle games.

We note that for all groups, there are some payoff parameters for which ${\delta }^F\to 1$, except in Group V where ${\delta }^F=0$ for all payoffs. For example, ${\delta }^F\to 1$ when $\mathfrak{b}\to -\infty$ for prisoner’s dilemma, stag hunt and coordination games. This means that we cannot extend the folk theorem unless these extreme payoff parameters can be ruled out.

We can observe that the value of ${\delta }^F$ depends on how large $V^{\ast }$ is compared to $V^{†}$. When $V^{\ast }$ is small, it is difficult to play certain actions in the game and the value of ${\delta }^F$ is high. For example, it is difficult to play the actions b and c in a prisoner’s dilemma when $\mathfrak{b}\to -\infty$. Geometrically, this means that $V^{\ast }$ stays almost the same but $V^{†}$ keeps increasing, making the proportion of $V^{\ast }$ to $V^{†}$ smaller. On the other hand, if $V^{\ast }$ is large then ${\delta }^F$ is smaller. Please note that Groups IV and V are exceptions, where ${\delta }^F$ is a constant and thus independent of $V^{\ast }$ and $V^{†}$.

We also note that there are discontinuities in ${\delta }^F$ on some of the group boundaries; see the thick grey lines in Figure 3. The discontinuity means that small changes in the payoffs can affect dramatically the payoff set and how large ${\delta }^F$ is. The discontinuity between the prisoner’s dilemma games is surprising and it shows that a small geometric change from the triangle-shape to the quadrangle-shape may affect whether the Pareto efficient payoffs are obtained or not.

For mixed strategies, we have a few remarks. The necessary and sufficient conditions for ${\delta }^F$ are more difficult to prove, since the strategies and their payoffs are more complicated. For Groups IIb and III, the values are only upper bounds since we only use the sufficient conditions. In all leader, battle of the sexes, coordination and anti-coordination games, ${\delta }^F=1$ but in the figure we use the values for which the pure-strategy FIR payoffs are obtained as a comparison.

4. Group I in Pure Strategies without Public Correlation

This group is defined by parameters $1<\mathfrak{c}<2-\mathfrak{b}$ and $\mathfrak{b}<0$. These are the prisoner’s dilemmas where $V^{\ast }$ is a quadrilateral with an obtuse angle in $u\left(a\right)$ corner; see Figure 2a. It is enough to examine the upper half of $V^{\ast }$ where $v_2>v_1$ since $V^{\ast }$ is symmetric with respect to the center line. Thus, the last point $v^F$ that is covered is only defined in the upper half of $V^{\ast }$.

The point $v_I^F$ in Group I is located on the upper edge between $u\left(a\right)$ and $u\left(b\right)$. This point and the corresponding discount factor ${\delta }_I^F$ are solved from the intersection of $B_a$ and $B_b$. It is enough to consider only player 1’s payoffs:

$$\begin{array}{ccc}\hfill v_I^F& =& v^B\left(B_a\left({\delta }^F\right)\right)=v^A\left(B_b\left({\delta }^F\right)\right)\hfill \\ & \Rightarrow & (1-{\delta }^F)\mathfrak{c}+{\delta }^F·0=(1-{\delta }^F)\mathfrak{b}+{\delta }^F·1\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \Rightarrow {\delta }_I^F& =& \frac{\mathfrak{c}-\mathfrak{b}}{1+\mathfrak{c}-\mathfrak{b}},\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill v_I^F& =& \left(\frac{\mathfrak{c}}{1-\mathfrak{b}+\mathfrak{c}},\frac{\mathfrak{b}-2\mathfrak{c}}{\mathfrak{b}-\mathfrak{c}-1}\right).\hfill \end{array}$$

(17)

On the second line, the first payoff $v_1^B\left(B_a\left({\delta }^F\right)\right)$ can be solved from the right-hand side of the admissibility condition.

We first show that $v^F\notin B_a^{\delta }\left(V^{\ast }\right)$, $a\in A$, if $\delta <{\delta }^F$. Since the sets $B_a$ and $B_b$ intersect at ${\delta }^F$, it means that $v^F$ does not belong to either $B_a$ or $B_b$ for $\delta <{\delta }^F$. Moreover, the sets $B_c$ and $B_d$ cannot cover $v^F$ on the boundary of $V^{\ast }$ by Proposition 2.

Now, let us show that every $v=\left\{v_2\ge v_1,v\in V^{\ast }\right\}\in B_a^{\delta }\left(V^{\ast }\right)$ for some $a\in A$ and $\delta ={\delta }^F$. We divide $V^{\ast }$ into three regions. If $v_1\ge v^M\left({\delta }^F\right)$ then $v\in B_a\left({\delta }^F\right)$ since $v^M\left(\delta \right)$ coincides with $v_1^{-}\left(B_a\left(V^{\ast }\right)\right)$ for all $\delta \ge {\delta }^F$ by Remark 1. If $v_1\le v^M\left({\delta }^F\right)$ and $v_2\le v^M\left({\delta }^F\right)$ then we show that $v\in B_d\left({\delta }^F\right)$. We show that all corner points $(v^M\left({\delta }^F\right),v^M\left({\delta }^F\right))$, $(0,0)$ and $(0,v^M\left({\delta }^F\right))$ belong to $B_d$. $(0,0)$ is the Nash equilibrium and belongs to $B_d$ for any $\delta$. Also, $(0,v^M\left({\delta }^F\right))$ belongs to $B_d$ if $(v^M\left({\delta }^F\right),v^M\left({\delta }^F\right))$ belongs to $B_d$, because $v_2^B\left(B_d\right)$ is higher than $v_2^A\left(B_d\right)$ due to $\mathfrak{c}>1$. So for the last corner, we need to show that $v^s\left(V_d\left({\delta }^F\right)\right)\ge v^M\left({\delta }^F\right)$. It holds that $v^s\left(V_d\left({\delta }^F\right)\right)={\delta }^F$ and $v^M\left({\delta }^F\right)=(1-{\delta }^F)\mathfrak{c}$ and the above condition holds if ${\delta }^F\ge (1-{\delta }^F)$. Using Equation (16), this is equal to $\mathfrak{b}\le 0$ and this is true in this group.

Finally, if $v_1\le v^M\left({\delta }^F\right)$ and $v_2\ge v^M\left({\delta }^F\right)$ then we show that $v\in B_b\left({\delta }^F\right)$. The corner points are $v^B\left(V^{\ast }\right)$, $(0,v^M\left({\delta }^F\right))$, $(v^M\left({\delta }^F\right),v^M\left({\delta }^F\right))$ and $v^B\left(B_a\left({\delta }^F\right)\right)$. $v^B\left(B_a\right)$ belongs to $B_b$ trivially by the definition of $v^F$. $v^B\left(V^{\ast }\right)\in B_b$ since $v^F$ is at the upper edge and $B_b$ covers the edge all the way to $v^B\left(V^{\ast }\right)$. $(0,v^M\left({\delta }^F\right))\in B_b$ since $v^M\left(\delta \right)$ coincides with $v_2^{-}\left(B_b\left(V^{\ast }\right)\right)$ for all $\delta \ge {\delta }^F$ when $\mathfrak{c}>\mathfrak{b}$ by Remark 1. Again, $\mathfrak{c}>1$ ensures that $v_1^C\left(B_b\right)>v_1^A\left(B_b\right)$ and thus $(v^M\left({\delta }^F\right),v^M\left({\delta }^F\right))\in B_b$ since $v_1^A\left(B_b\left({\delta }^F\right)\right)=v_1^F=v^M\left({\delta }^F\right)$.

5. Conclusions

This paper examines the discount factor values for which the subgame-perfect equilibrium payoffs coincide with the FIR payoffs in the symmetric $2\times 2$ supergames. The main motivation is to study if the folk theorem could be extended in a class of games and find out the reasons why a high level of patience such as $\delta \to 1$ is required in some games. We find that the main reason is that it is impossible to obtain payoffs close to the minimax values: (1) this happens in Group IV in all strategies, (2) it is a result of the fact that the mixed-strategy punishment payoff is strictly smaller than the pure-strategy punishment in leader, battle of the sexes, coordination and its anti-game in mixed strategies, and (3) it is due to geometrical reasons in some games, i.e., how the stage-game payoffs are located and how large the FIR payoffs $V^{\ast }$ are in proportion to feasible payoffs $V^{†}$. If $V^{\ast }$ is small, which also means that it is difficult to play certain actions in the game, then a high level of patience is required.

We also examine how the public correlation and the mixed strategies affect the results. The games are organized into a few groups based on the equation that determines the smallest discount factor value as a function of the stage-game payoffs. The groups and the equations are a bit different under different strategies, but the overview looks similar. Even though a lower level of patience is required with public randomization, the highest and the lowest values are obtained in the same regions. The limit ${\delta }^F\to 1$ is obtained when $b\to -\infty$, when $c\approx b$, when $b\ge 1$ and $b+c\approx 2$, or in certain anti-games. Thus, it is not possible to extend the folk theorem in any of the typical game classes, such as prisoner’s dilemma, chicken and stag hunt games, unless certain extreme payoffs can be ruled out. Furthermore, the public correlation or mixed strategies do not provide a remedy in these games and it holds that ${\delta }^F\to 1$ under all strategies.

The results of this paper help in determining the payoff set for high discount factor values. If the discount factor is higher than ${\delta }^F$, then all the FIR payoffs are subgame-perfect equilibrium payoffs. Moreover, it is a bit surprise how small ${\delta }^F$ can be with public correlation. It was also observed that there are certain boundaries where ${\delta }^F$ is discontinuous, which means that small changes in the payoffs may affect dramatically the equilibrium payoffs.

It should be noted that this kind of analysis could be done in asymmetric games with more than two actions and more than two players. Furthermore, it is left for future research how to compute efficiently all the equilibrium payoffs when the discount factors are smaller than ${\delta }^F$.