Indirect Reciprocity with Optional Interactions and Private Information

Abstract

We consider indirect reciprocity with optional interactions and private information. A game is offered between two players and accepted unless it is known that the other person is a defector. Whenever a defector manages to exploit a cooperator, his or her reputation is revealed to others in the population with some probability. Therefore, people have different private information about the reputation of others, which is a setting that is difficult to analyze in the theory of indirect reciprocity. Since a defector loses a fraction of his social ties each time he exploits a cooperator, he is less efficient at exploiting cooperators in subsequent rounds. We analytically calculate the critical benefit-to-cost ratio above which cooperation is successful in various settings. We demonstrate quantitative agreement with simulation results of a corresponding Wright–Fisher process with optional interactions and private information. We also deduce a simple necessary condition for the critical benefit-to-cost ratio.

1. Introduction

The evolution of human cooperation is an intensely researched topic in the biological and economic sciences [1,2,3,4]. In the classic prisoner’s dilemma game, the rational choice for either player is to defect regardless of the other player’s choice of strategy. Even in the repeated prisoner’s dilemma with exactly m rounds, defection on every round is the only strict Nash equilibrium strategy and the only evolutionarily stable strategy. Indirect reciprocity is an important mechanism for the evolution of cooperation [1]. The basic setting of indirect reciprocity is repeated interactions in a group of players [5,6,7]. My behavior toward you depends on what you have done to me and to others [8,9,10].

Indirect reciprocity is a generalization of direct reciprocity [1]. Direct reciprocity is the phenomenon in which a person behaves toward another individual based on prior personal experience with that individual. For example, if Individual A had a positive interaction with Individual B, then Individual A would be more inclined to cooperate with Individual B in subsequent encounters. Indirect reciprocity works via reputation [11,12,13]. For example, if Individual A had a positive interaction with Individual B, then Individual B would achieve a good reputation also in the eyes of an observer, Individual C. Individual C would remember Individual B’s good score and be more likely to cooperate with Individual B in subsequent encounters. In this way, people help those who help others, and helpful people have a higher payoff in the end, as is shown in experimental studies [14,15,16,17,18,19,20,21,22]. One sees that indirect reciprocity can foster cooperation in the long run because cooperators are able to channel benefits primarily toward other cooperators.

Many theoretical studies have focused primarily on the case of public information [23,24,25]. In this setting, all individuals in the population have the same opinion about the reputation of any particular individual. Various subtopics have been researched, including the effects of image scoring and good standing strategies [23], the reputation dynamics that lead to evolution of indirect reciprocity [24,25], involuntary defection [26], games among more than two players [27,28], the ability of cheaters to disrupt stable strategies [29], costly information transfer [30], trinary (instead of binary) reputation models [31], mixing of social norms [32], and others. Public information is a nice modeling simplification, and it can serve as a powerful promoter for the evolution and maintenance of cooperation.

But the original and more general formulation of indirect reciprocity allows for private information, where players differ in their opinion concerning the reputation of others [33,34,35,36,37,38,39]. Some of the original studies of indirect reciprocity via image scoring have explored the evolutionary dynamics with private information [38,39]. More recent studies have discussed topics of indirect reciprocity with private information, such as the effects of private information on assessment rules [40,41], incomplete observation [42], and assessment errors [43], among others. Importantly, the distribution of reputations in the population, which specifies how individuals perceive each other, can be evolving over time. Basic analytical results with private information—and with an evolving distribution of reputations—are typically difficult to achieve.

2. Model

We investigate a simple model of indirect reciprocity with private reputation. The identity of a defector may be known to some people, but not to others. We consider a prisoner’s dilemma with two strategies: cooperation, C, and defection, D. The payoff matrix is given by

$$\begin{array}{cccc}& & C& D\\ C& (& b-c& -c& )\\ D& b& 0\end{array}$$

Consider a population of $N=N_C+N_D$ individuals, in which $N_C$ individuals are cooperators, and $N_D$ individuals are defectors. In a single round, two individuals are chosen randomly, and they are offered a game. A cooperator always plays with a cooperator, and each receives payoff $b-c$. If a cooperator and a defector are chosen in a round, then the cooperator accepts to play the game only if he or she is unaware of the defector’s identity. If the cooperator is cognizant of the defector’s identity, then the potential game is rejected. Thus, our model features optional interactions between players [44]. If a game is played between an unknowing cooperator and a defector, then the cooperator receives payoff $-c$, while the defector receives payoff b. If two defectors are chosen in a round, then they play, and each receives payoff 0. (Alternately, one could consider that two defectors simply do not play a game together; since a game between two defectors has no effect on payoffs and results in no information transfer, such a distinction is inconsequential for our model.) Each time a defector plays a game with a cooperator, his or her identity is revealed to each of the $N_C$ cooperators in the population with probability p. We assume that this information transfer about the identity of the defector between cooperators is not costly [30].

A total of M rounds occur in a single generation. We use k to denote one of the N individuals in the population. The payoffs from all games of individual k over M rounds are added to obtain that individual’s total payoff, $S_k$. Individual k’s fitness is computed as $F_k=exp\left(\beta S_k\right)$, where β represents the intensity of selection. After M rounds, Wright–Fisher updating is performed. For each individual in the next generation, a parent is chosen from the current generation proportional to fitness. Mutation occurs with probability μ and leads equiprobably either to a cooperator or to a defector. Then M rounds are played in the new generation. Payoff does not accumulate from one generation to the next, but is always reset to zero.

What is the critical value of $b/c$ above which cooperators are more abundant than defectors when their numbers, $N_C$ and $N_D$, are averaged over many successive generations of the mutation–selection dynamics? What is the critical value of $b/c$ above which, with no mutation, defection fixes in the population with probability less than $1/N$ when starting with a single defector? Can these questions be answered analytically?

3. Results

3.1. Single Defector

To make progress analytically, imagine the simpler case of a single defector in an infinitely large population of cooperators. In this simplified setting, the defector participates in m rounds and therefore is offered to play a game m times, each time with a random cooperator. (Note that in the dynamical model, M rounds are run in total between randomly selected pairs of individuals, but the number of rounds that a particular defector in the population participates in is a random variable. In the simplified static model, the sole defector participates in exactly m rounds.) After each exploitation of a cooperator, the exploited cooperator shares the identity of the solitary defector with each of the other uninformed cooperators with probability $p=1-q$. Thus, with probability q, each defector–cooperator link remains active after an exploitation of a cooperator by a defector. Informed cooperators always refuse a potential game with the defector. The fraction of cooperators in the population that are aware of the defector’s identity increases with the number of games played. Since informed cooperators always reject a game with the defector, cooperators can achieve a higher total payoff than the defector if many rounds are offered. A sample sequence of rounds is shown in Figure 1. Changes in the fraction of active links to cooperators as games are played are shown schematically in Figure 2.

Figure 1. Schematic illustrating indirect reciprocity with optional interactions and private information in a population with eight cooperators (blue) and a single defector (red). The population size is $N=9$. The defector is at the center of each image of the population. Each solid black line connecting the defector with a cooperator represents an active link. In the first round, the defector is offered a game with a random cooperator (light blue with a “?”), and the game is played. The defector gains payoff b. Each of the defector’s links is removed with probability $p=0.5$. After the first round, four of the defector’s links with cooperators are eliminated (thin dotted lines). In the second round, the defector is offered a game with a cooperator that knows the defector’s identity, so the potential game is rejected. In the third round, the defector again exploits an ignorant cooperator, and the defector again receives payoff b. Two of the remaining active links are eliminated. In the fourth round, the defector is offered a game with an informed cooperator, and the potential game is rejected. After this sequence of four rounds, the defector gains total payoff $2b$.

Figure 1. Schematic illustrating indirect reciprocity with optional interactions and private information in a population with eight cooperators (blue) and a single defector (red). The population size is $N=9$. The defector is at the center of each image of the population. Each solid black line connecting the defector with a cooperator represents an active link. In the first round, the defector is offered a game with a random cooperator (light blue with a “?”), and the game is played. The defector gains payoff b. Each of the defector’s links is removed with probability $p=0.5$. After the first round, four of the defector’s links with cooperators are eliminated (thin dotted lines). In the second round, the defector is offered a game with a cooperator that knows the defector’s identity, so the potential game is rejected. In the third round, the defector again exploits an ignorant cooperator, and the defector again receives payoff b. Two of the remaining active links are eliminated. In the fourth round, the defector is offered a game with an informed cooperator, and the potential game is rejected. After this sequence of four rounds, the defector gains total payoff $2b$.

Figure 2. Schematic showing how the fraction of active links between the defector and cooperators decreases as additional exploitations occur. At the start of a generation, all links to cooperators are active. On the first round, the defector plays a game, and a fraction q of the links remain active after the game. On each round thereafter, the defector plays another game with probability $q^i$, where i is the number of games that have been played to that point.

Figure 2. Schematic showing how the fraction of active links between the defector and cooperators decreases as additional exploitations occur. At the start of a generation, all links to cooperators are active. On the first round, the defector plays a game, and a fraction q of the links remain active after the game. On each round thereafter, the defector plays another game with probability $q^i$, where i is the number of games that have been played to that point.

We denote the average number of games played in m rounds by a solitary defector in a population of infinitely many cooperators by $F_m\left(q\right)$. We must find an expression for $F_m\left(q\right)$. First, let us derive an expression for $x_{i,m}$—the probability that the defector plays i games in m rounds. Suppose that the defector plays i times in m rounds. There are two possible cases: (i) the defector plays i times in the first $m-1$ rounds and is rejected for play in round m; and (ii) the defector plays $i-1$ times in the first $m-1$ rounds and is accepted for play in round m. The probability for the first case is $(1-q^i)x_{i,m-1}$. The probability for the second case is $q^{i-1}{x}_{i-1,m-1}$. Therefore, the probability that the defector plays i times in m rounds is

$$x_{i,m}=(1-q^i)x_{i,m-1}+q^{i-1}{x}_{i-1,m-1}$$

(1)

This expression holds for any $m\ge i\ge 2$ with the convention that $x_{m,m-1}=0$. For $i=1$, the second case does not occur and we have the recurrence formula

$$x_{1,m}=(1-q)x_{1,m-1}\forall m\ge 2$$

Since $x_{1,1}=1$, we have

$$x_{1,m}={(1-q)}^{m-1}\forall m\ge 1$$

(2)

3.1.1. Probability of i Games in m Rounds

To solve the recurrence, Equation (1), for $x_{i,m}$, it is helpful to first solve for its z-transform, which we write as

$${\tilde{x}}_i\left(z\right)=\sum _{m=0}^{\infty }{z}^{-m}{x}_{i,m}$$

(3)

From Equation (3) and Equation (2), the z-transform of $x_{1,m}$ is

$${\tilde{x}}_1\left(z\right)=\frac{1}{z}\sum _{m=0}^{\infty }{\left(\frac{1-q}{z}\right)}^m$$

We can consider the summation as a geometric series (with region of convergence $\left|z\right|>1-q$), and we obtain

$${\tilde{x}}_1\left(z\right)=\frac{1}{z-(1-q)}$$

(4)

From Equation (3) and Equation (1), the z-transform of $x_{i,m}$ with $i\ge 2$ is

$${\tilde{x}}_i\left(z\right)=(1-q^i)z^{-1}{\tilde{x}}_i\left(z\right)+q^{i-1}{z}^{-1}{\tilde{x}}_{i-1}\left(z\right)$$

We see that ${\tilde{x}}_i\left(z\right)$ can be expressed recursively in terms of ${\tilde{x}}_{i-1}\left(z\right)$:

$${\tilde{x}}_i\left(z\right)=\frac{q^{i-1}}{z-(1-q^i)}{\tilde{x}}_{i-1}\left(z\right)$$

(5)

Iterating Equation (5) and substituting Equation (4), we find

$${\tilde{x}}_i\left(z\right)=\prod _{n=1}^i\frac{q^{n-1}}{z-(1-q^n)}$$

(6)

Thus, the z-transformed quantity ${\tilde{x}}_i\left(z\right)$ has a simple, closed-form expression in terms of q, i, and z. All that remains is to invert the z-transform to obtain $x_{i,m}$. Intuitively, Equation (6) has exactly i simple poles, and each simple pole corresponds to a separate term in the evaluation of $x_{i,m}$. We have

$$x_{i,m}=\frac{1}{2\pi I}{\oint }_{C}dz{z}^{m-1}{\tilde{x}}_i\left(z\right)=\frac{1}{2\pi I}{\oint }_{C}dz{z}^{m-1}\prod _{n=1}^i\frac{q^{n-1}}{z-(1-q^n)}$$

(I denotes the imaginary unit.) Here, C represents a counterclockwise closed contour encircling the origin and enclosing all poles of the integrand. For example, we can take a unit circle for C as all the zeros and poles have modulus less than 1. From the residue theorem, we have

$$x_{i,m}=\sum _{k=1}^i\underset{z=1-q^k}{\mathrm{Res}}\frac{z^m}{z}\prod _{n=1}^i\frac{q^{n-1}}{z-(1-q^n)}$$

We evaluate this as

$$x_{i,m}=\sum _{k=1}^i\underset{z\to 1-q^k}{lim}[z-(1-q^k)]\frac{z^m}{z}\prod _{n=1}^i\frac{q^{n-1}}{z-(1-q^n)}$$

(7)

Simplifying Equation (7), we obtain

$$x_{i,m}=\sum _{k=1}^i{(-1)}^{i-k}{q}^{i(i-1)/2}\frac{{(1-q^k)}^m}{1-q^k}\left[\prod _{n=1}^{k-1}\frac{1}{q^n(1-q^n)}\right]\left[\prod _{n=1}^{i-k}\frac{1}{q^k(1-q^n)}\right]$$

This can be rearranged slightly as

$$x_{i,m}=\sum _{k=1}^i{(-1)}^{i-k}{q}^{k+i(i-1)/2}{(1-q^k)}^m\left[\prod _{n=1}^k\frac{1}{q^n(1-q^n)}\right]\left[\prod _{n=1}^{i-k}\frac{1}{q^k(1-q^n)}\right]$$

Collecting factors of q, we have

$$x_{i,m}=\sum _{k=1}^i{(-1)}^{i-k}{q}^{(i-k)(i-k-1)/2}{(1-q^k)}^m\left[\prod _{n=1}^k\frac{1}{1-q^n}\right]\left[\prod _{n=1}^{i-k}\frac{1}{1-q^n}\right]$$

(8)

We make the substitution $i=j+k$. We also define

$$Q_k=\prod _{i=1}^k\frac{1}{1-q^i}$$

Equation (8) can then be simplified as

$$x_{i,m}=\sum _{j=0}^{i-1}{(-1)}^j{q}^{j(j-1)/2}{(1-q^{i-j})}^m{Q}_{i-j}{Q}_j$$

(9)

Equation (9) is proven using an alternative method in Appendix A.

3.1.2. Average Number of Games

The average number of games played by the defector, $F_m\left(q\right)$, is given by

$$F_m\left(q\right)=\sum _{i=1}^{m}i{x}_{i,m}$$

(10)

From Equation (9) and Equation (10), and after rewriting the summations, we have

$$F_m\left(q\right)=\sum _{j=1}^m{(1-q^j)}^m{Q}_j\sum _{k=0}^{m-j}{(-1)}^k(j+k)q^{k(k-1)/2}{Q}_k$$

(11)

In Figure 3, we plot Equation (11) in several ways.

3.2. Fixed Number of Rounds per Generation

More generally, consider that there are $N_C$ cooperators and $N_D$ defectors. Two players are selected randomly in each round and are offered a game. Each individual is selected, on average, m times in M total rounds. The total number of rounds, M, is a constant for each generation.

3.2.1. Average Number of Games

Define by $f=N_C/N$ the fraction of individuals in the population that are cooperators. A total of M rounds are run in each generation in a population of size N. Here, M can be expressed in terms of m (the average number of rounds that each individual engages in) as $M=mN/2$. The probability that a given defector engages in a particular round is $2/N$. In the limit $N\to \infty$, the binomial distribution for the number of rounds that a given defector engages in becomes Poisson-distributed with expected value $(mN/2)(2/N)=m$. Moreover, the average number of times that a defector is selected for a possible game with a cooperator is $fm$, and this quantity is Poisson-distributed with expected value $fm$. Thus, we can write the average number of games played by a defector with cooperators in a generation as

$$G(f,m;p)=e^{-fm}\sum _{k=1}^{\infty }\frac{f^k{m}^k}{k!}{F}_k(1-p)$$

Here, $p=1-q$ is the probability that a link between a cooperator and a defector is eliminated after the defector exploits a cooperator.

Figure 3. Average number of games that the defector plays. (a) Expected number of exploitations, $F_m\left(q\right)$, against the probability, q, to maintain each link when $m=5$, 10, and 20 (bottom to top); (b) $F_m\left(q\right)$ against the number of rounds, m, when $q=0.2$, $0.5$, and $0.8$ (bottom to top); (c) Expected number of exploitations normalized by the number of rounds, $F_m\left(q\right)/m$, against the probability, q, to maintain each link when $m=5$, 10, and 20 (top to bottom); (d) $F_m\left(q\right)/m$ against the number of rounds, m, when $q=0.2$, $0.5$, and $0.8$ (bottom to top).

Figure 3. Average number of games that the defector plays. (a) Expected number of exploitations, $F_m\left(q\right)$, against the probability, q, to maintain each link when $m=5$, 10, and 20 (bottom to top); (b) $F_m\left(q\right)$ against the number of rounds, m, when $q=0.2$, $0.5$, and $0.8$ (bottom to top); (c) Expected number of exploitations normalized by the number of rounds, $F_m\left(q\right)/m$, against the probability, q, to maintain each link when $m=5$, 10, and 20 (top to bottom); (d) $F_m\left(q\right)/m$ against the number of rounds, m, when $q=0.2$, $0.5$, and $0.8$ (bottom to top).

3.2.2. Critical Benefit-to-Cost Ratio

What is the critical benefit-to-cost ratio, $b/c$, needed to ensure that cooperators receive a higher payoff, on average, than defectors? We reason as follows: Each cooperator plays an average of $fm$ games with other cooperators in a generation, so each cooperator receives a contribution to its payoff in the amount $fm(b-c)$. Each defector plays, on average, $G(f,m;p)$ games in a generation, so each defector receives average payoff $bG(f,m;p)$. Each exploitation by a defector corresponds to a cost incurred by a cooperator without any compensating benefit. The total number of games that are played by defectors, on average, is $N(1-f)G(f,m;p)$. Therefore, the average cost incurred by a cooperator due to exploitations by defectors is $(-c)\left(\right(N(1-f))/(Nf\left)\right)G(f,m;p)$. Putting these pieces together, we find that the average cooperator payoff in a generation exceeds the average defector payoff in a generation if

$$fm(b-c)-c\frac{1-f}{f}G(f,m;p)>bG(f,m;p)$$

(12)

Rearranging Equation (12), we find that cooperators achieve a higher payoff that defectors, on average, if

$$\frac{b}{c}>1+{\left[f\left(\frac{fm}{G(f,m;p)}-1\right)\right]}^{-1}\equiv A(f,m;p)$$

(13)

3.2.3. Simulation Results

We next perform simulations for which the total number of rounds per generation, M, is fixed. The success of cooperation depends on the rate, p, at which information about the defectors is transferred among cooperators, on the fraction, f, of cooperators in the population, and on the average number of rounds per generation, m, that an individual engages in. We have that $m=2M/N$.

Simulation results for a fixed total number of rounds per generation are shown in Figure 4a,b. In Figure 4a, the total number of rounds per generation, M, is 400. Therefore, the average number of rounds per individual in a generation, m, is equal to 8. In Figure 4b, the total number of rounds per generation, M, is 800. Therefore, the average number of rounds per individual in a generation, m, is equal to 16.

We plot the critical value of $b/c$ needed for cooperators to be more abundant than defectors with nonzero mutation rate, μ, and with weak intensity of selection, β (“+” symbols), for $m=8$ and $m=16$. We also plot the critical value of $b/c$ above which cooperators achieve a higher payoff in M total rounds, on average, than do defectors if $f=1/2$ (green circles) for $m=8$ and $m=16$. The two sets of data points agree well, as we would expect. In addition, we show the functions $A(1/2,8;p)$ and $A(1/2,16;p)$. The agreement between theory and simulation is excellent.

In another measurement, we start with a single defector and allow the dynamics to progress with no mutation ($\mu =0$) and with weak intensity of selection. We plot the critical value of $b/c$ needed for the fixation probability at the all-defector state to be less than $1/N$ (“X” symbols) for $m=8$ and $m=16$ when starting with a single defector. We also plot the critical value of $b/c$ above which cooperators achieve a higher payoff in M total rounds, on average, than do defectors if $f=2/3$ (red squares) for $m=8$ and $m=16$. The two sets of data points agree well, as we would expect from the one-third law of evolutionary dynamics [2,47,48,49,50,51]. In addition, we show the functions $A(2/3,8;p)$ and $A(2/3,16;p)$. The agreement between theory and simulation is again excellent.

In yet another measurement, we plot the critical value of $b/c$ above which a solitary defector in a population of cooperators achieves a lower payoff in M total rounds, on average, than does a random cooperator (black triangles). This is the condition for cooperation to be a strict Nash equilibrium. We also show the functions $A(1,8;p)$ and $A(1,16;p)$, again demonstrating outstanding agreement between theory and simulation.

Figure 4. The critical value of $b/c$ for cooperation to be favored over defection in various settings. The green circles represent measured critical values of $b/c$ for the average cooperator payoff to exceed the average defector payoff when $f=1/2$. The “+” symbols represent measured critical values of $b/c$ for cooperators to outnumber defectors with weak selection and with weak mutation when their numbers are averaged over many successive generations. The red squares represent measured critical values of $b/c$ for the average cooperator payoff to exceed the average defector payoff when $f=2/3$. The “×” symbols represent measured critical values of $b/c$ for defectors to fix with probability less than $1/N$ when starting with a single defector with weak selection and with no mutation. The black triangles represent measured critical values of $b/c$ for the average cooperator payoff to exceed the average defector payoff when $f=1$ (i.e., a single defector in a large population of cooperators). Panels (a,b) are for a fixed total number of rounds per generation, M, while panels (c,d) are for a variable total number of rounds per generation, M, where the average of M over generations is $\overline{M}=1/(1-W)$. Error bars are roughly the size of the data points.

Figure 4. The critical value of $b/c$ for cooperation to be favored over defection in various settings. The green circles represent measured critical values of $b/c$ for the average cooperator payoff to exceed the average defector payoff when $f=1/2$. The “+” symbols represent measured critical values of $b/c$ for cooperators to outnumber defectors with weak selection and with weak mutation when their numbers are averaged over many successive generations. The red squares represent measured critical values of $b/c$ for the average cooperator payoff to exceed the average defector payoff when $f=2/3$. The “×” symbols represent measured critical values of $b/c$ for defectors to fix with probability less than $1/N$ when starting with a single defector with weak selection and with no mutation. The black triangles represent measured critical values of $b/c$ for the average cooperator payoff to exceed the average defector payoff when $f=1$ (i.e., a single defector in a large population of cooperators). Panels (a,b) are for a fixed total number of rounds per generation, M, while panels (c,d) are for a variable total number of rounds per generation, M, where the average of M over generations is $\overline{M}=1/(1-W)$. Error bars are roughly the size of the data points.

3.3. Variable Number of Rounds per Generation

The only strict Nash equilibrium strategy and the only evolutionarily stable strategy in the classic prisoner’s dilemma with exactly m repeated interactions is to defect in every round [2,12,52]. In realistic settings, an individual may not know at the outset how many rounds it will participate in with potential interaction partners. It is therefore worthwhile to also consider a variable number of rounds per generation. In a variation of our dynamical model, we postulate that another round is run between two randomly chosen individuals with probability W. The average total number of rounds that are run is $\overline{M}=1/(1-W)$. The average number of rounds that each defector engages in, averaged over generations, is $\overline{m}$.

3.3.1. Average Number of Games

The probability that a total of M rounds are run is $(1-W)W^{M-1}$. Analogously to $G(f,m;p)$, we can write the average number of games played by a defector, $H(f,\overline{m};p)$, as

$$H(f,\overline{m};p)=\underset{N\to \infty }{lim}(1-W)\sum _{M=1}^{\infty }{W}^{M-1}{e}^{\frac{-2fM}{N}}\sum _{k=1}^{\infty }\frac{2^k{f}^k{M}^k}{N^{k}k!}{F}_k(1-p)$$

Intuitively, we are just multiplying the probability that a total of M rounds are offered in a generation with the average number of rounds offered to an individual, $G(f,m=2M/N;p)$. We consider the limit of large population size, $N\to \infty$, which is realistic and also admits simplified results. Denote the average number of rounds that an individual engages in by $\overline{m}$. The value of W for which the average number of rounds engaged in per individual equals $\overline{m}$ is $W=1-2/\left(\overline{m}N\right)$. We have

$$H(f,\overline{m};p)=\underset{N\to \infty }{lim}\frac{2}{\overline{m}N}\sum _{M=1}^{\infty }{\left(1-\frac{2}{\overline{m}N}\right)}^{M-1}{e}^{\frac{-2fM}{N}}\sum _{k=1}^{\infty }\frac{2^k{f}^k{M}^k}{N^{k}k!}{F}_k(1-p)$$

We can write this as

$$H(f,\overline{m};p)=\underset{N\to \infty }{lim}\frac{2}{\overline{m}N}\sum _{M=1}^{\infty }{\left[{\left(1-\frac{2}{\overline{m}N}\right)}^{\frac{\overline{m}N}{2}}\right]}^{\frac{2(M-1)}{\overline{m}N}}{e}^{\frac{-2fM}{N}}\sum _{k=1}^{\infty }\frac{2^k{f}^k{M}^k}{N^{k}k!}{F}_k(1-p)$$

In the limit of large population size, $N\to \infty$, the factor in square brackets becomes $e^{-1}$. Thus, we have

$$H(f,\overline{m};p)=\underset{N\to \infty }{lim}\frac{2}{\overline{m}N}\sum _{M=1}^{\infty }{e}^{\frac{-2M}{\overline{m}N}}{e}^{\frac{-2fM}{N}}\sum _{k=1}^{\infty }\frac{2^k{f}^k{M}^k}{N^{k}k!}{F}_k(1-p)$$

This can be rewritten more suggestively as

$$H(f,\overline{m};p)=\frac{1}{\overline{m}}\underset{N\to \infty }{lim}\sum _{M=1}^{\infty }\frac{2\Delta M}{N}{e}^{-\left(\frac{1+f\overline{m}}{\overline{m}}\right)\frac{2M}{N}}\sum _{k=1}^{\infty }\frac{f^k}{k!}{\left(\frac{2M}{N}\right)}^k{F}_k(1-p)$$

(14)

Here, note that $\Delta M=1$. (The summation increments M by 1 at each iteration.) Recall that $m=2M/N$. Since $2\Delta M/N=\Delta m$, we can consider Equation (14) as an integration over m. We have

$$H(f,\overline{m};p)=\frac{1}{\overline{m}}{\int }_0^{\infty }dm{e}^{-\left(\frac{1+f\overline{m}}{\overline{m}}\right)m}\sum _{k=1}^{\infty }\frac{f^k}{k!}{m}^k{F}_k(1-p)$$

Next, we define

$$\zeta =\left(\frac{1+f\overline{m}}{\overline{m}}\right)m$$

We have

$$H(f,\overline{m};p)=\frac{1}{1+f\overline{m}}\sum _{k=1}^{\infty }{\left(\frac{\overline{m}}{1+f\overline{m}}\right)}^k\left({\int }_0^{\infty }d\zeta e^{-\zeta }{\zeta }^k\right)\frac{f^k}{k!}{F}_k(1-p)$$

Notice that the integration over ζ is just $\Gamma (k+1)=k!$. Thus, we have

$$H(f,\overline{m};p)=\frac{1}{1+f\overline{m}}\sum _{k=1}^{\infty }{\left(\frac{f\overline{m}}{1+f\overline{m}}\right)}^k{F}_k(1-p)$$

3.3.2. Critical Benefit-to-Cost Ratio

Analogously to Equation (12), the average cooperator payoff in a generation exceeds the average defector payoff in a generation if

$$f\overline{m}(b-c)-c\frac{1-f}{f}H(f,\overline{m};p)>bH(f,\overline{m};p)$$

(15)

Rearranging Equation (15), we have

$$\frac{b}{c}>1+{\left[f\left(\frac{f\overline{m}}{H(f,\overline{m};p)}-1\right)\right]}^{-1}\equiv B(f,\overline{m};p)$$

(16)

3.3.3. Simulation Results

We next perform simulations for which the total number of rounds per generation, M, is a random variable. The success of cooperation depends on the rate, p, at which information about the defectors is transferred among cooperators, on the fraction, f, of cooperators in the population, and on the average number of rounds per generation, $\overline{m}$, that an individual engages in. The average number of rounds that an individual engages in during any single generation, m, is a random variable and is given by $m=2M/N$.

Simulation results for a variable total number of rounds per generation are shown in Figure 4c,d. In Figure 4c, the probability of another round, W, is $399/400$. Therefore, the average number of rounds per individual in a generation, $\overline{m}$, is equal to 8. We plot $B(1/2,8;p)$, $B(2/3,8;p)$, and $B(1,8;p)$. The agreement between theory and simulation is excellent. In Figure 4d, the probability of another round, W, is $799/800$. Therefore, the average number of rounds per individual in a generation, $\overline{m}$, is equal to 16. We plot $B(1/2,16;p)$, $B(2/3,16;p)$, and $B(1,16;p)$. The agreement between theory and simulation is again excellent.

Note the subtle distinction between the models investigated in Figure 4a,b and Figure 4c,d. In Figure 4a,b, the total number of rounds per generation is fixed and is equal to M, and the average number of rounds per individual in a generation is equal to $m=2M/N$. In Figure 4c,d, the probability of another round between two random individuals in a generation is equal to W. The total number of rounds per generation, M, for Figure 4c,d is therefore a random variable, with average given by $\overline{M}=1/(1-W)$, and the average number of rounds per individual in a generation is equal to $\overline{m}=2\overline{M}/N=2/\left(N(1-W)\right)$. Notice that the critical values of $b/c$ are lower in Figure 4c than in Figure 4a and are also lower in Figure 4d than in Figure 4b.

3.4. Simple Lower Bound on $b/c$

We now seek a simple condition on $b/c$ for cooperation to be successful. Consider an individual in an infinitely large population of cooperators. The individual participates in $m+1$ rounds with the infinite population of cooperators and has two choices of strategy: The focal individual may cooperate in all $m+1$ rounds, receiving total payoff $(b-c)(m+1)$. Or the focal individual may defect on the first round but cooperate on all m subsequent rounds, receiving total payoff $b+qm(b-c)$. The strategy for the focal individual to always cooperate delivers a higher payoff after $m+1$ rounds if

$$(b-c)(m+1)>b+qm(b-c)$$

(17)

We expect Equation (17) to yield a necessary condition on $b/c$ for cooperation to be successful. Rearranging, we arrive at the condition

$$\frac{b}{c}>1+\frac{1}{mp}$$

(18)

If the total number of rounds, M, in a generation is a random variable, then we propose a variation of Equation (18):

$$\frac{b}{c}>1+\frac{1}{\overline{m}p}$$

(19)

The simple condition, Equation (18), is plotted in Figure 4a,b (solid black curves). Notice that this form has the intuitively correct limiting behavior: The critical value of $b/c$ increases as the rate of information transfer, p, decreases, and becomes infinite in the limit $p\to 0$. The critical value of $b/c$ decreases as the number of rounds, m, increases. As $m\to \infty$, cooperators are able to accumulate an infinitely larger payoff than defectors in a single generation (provided that $b-c>0$), and the critical value of $b/c$ approaches 1. The simple condition, Equation (19), is also plotted in Figure 4c,d (solid black curves).

4. Discussion

We have studied a simple model for evolution of indirect reciprocity with private information. As information about the identity of defectors spreads among cooperators, defectors lose their fitness advantage. Interactions are optional; a cooperator can reject a possible game with a defector if it is cognizant of that defector’s identity. Thus, cooperation can eventually prevail.

We have derived exact conditions on the benefit-to-cost ratio for cooperation to dominate. We have also derived exact conditions on the benefit-to-cost ratio for natural selection to oppose the fixation of defectors starting with a single defector. Agreement of our theoretical predictions for either a fixed or a variable total number of rounds (Equation (13) and Equation (16), respectively) with simulation data is excellent. We have also deduced simple conditions on the benefit-to-cost ratio, Equation (18) and Equation (19), which ensure the success of cooperation.

Our calculations and simulations lend insight into indirect reciprocity with an evolving distribution of reputations. Increasing the rate of information transfer among cooperators and increasing the number of games offered to each individual in a generation both facilitate the transition to a population of cooperators. Intriguingly, the distribution of the total number of rounds in a generation significantly affects the critical benefit-to-cost ratio needed for cooperators to be favored. This is evident in Figure 4, where one sees that having a constant probability of another round in a generation (as opposed to having an exactly fixed total number of rounds per generation) is more favorable to cooperation.

Variations of our model are also possible. For example, a defector’s identity may be revealed in any round in which it is paired with a cooperator, even if the cooperator rejects the potential game. Notice that this modified model is significantly more conducive to the evolution of cooperation since information about defectors is transferred more rapidly. In this case, our critical benefit-to-cost ratios, Equation (13) and Equation (16), would represent sufficient conditions for cooperators to succeed. We derive results for the modified model in Appendix B.

In another variation, there is a cost for entering a game. In this case, even defectors would want to avoid playing with other defectors. It is now plausible that also defectors reveal the identities of other defectors that they have met. This modification also allows for easier evolution of cooperation, because the identity of defectors is revealed faster.

There are also possibilities for implementation errors. For example, when a cooperator engages in a round with a defector, the cooperator could mistake the defector for being another cooperator. Such recognition errors may involve new information transfer regarding the defector’s identity, which would reduce future exploitations of cooperators by that defector. Implementation errors could also result in erroneous information transfer if cooperators are mistakenly perceived as defectors. In this case, cooperators that are wrongly perceived as defectors might have their links with other cooperators severed for all subsequent rounds, which would make it more difficult for cooperators to thrive. However, a key point is that we endow cooperators with optional interactions. Therefore, on rare occasions, a cooperator might wrongfully view another cooperator as a defector and reject a potential game, but both individuals subsequently retain their strategies as cooperators. For this reason, we expect our derivations of critical benefit-to-cost ratios to be fairly robust provided that implementation errors are rare. Each of these modifications to our model is a potential topic for future study.