## Abstract

A key, yet often neglected, component of digital evolution and evolutionary models is the ‘selection method’ which assigns fitness (number of offspring) to individuals based on their performance scores (efficiency in performing tasks). Here, we study with formal analysis and numerical experiments the evolution of cooperation under the five most common selection methods (proportionate, rank, truncation-proportionate, truncation-uniform and tournament). We consider related individuals engaging in a Prisoner's Dilemma game where individuals can either cooperate or defect. A cooperator pays a cost, whereas its partner receives a benefit, which affect their performance scores. These performance scores are translated into fitness by one of the five selection methods. We show that cooperation is positively associated with the relatedness between individuals under all selection methods. By contrast, the change in the performance benefit of cooperation affects the populations’ average level of cooperation only under the proportionate methods. We also demonstrate that the truncation and tournament methods may introduce negative frequency-dependence and lead to the evolution of polymorphic populations. Using the example of the evolution of cooperation, we show that the choice of selection method, though it is often marginalized, can considerably affect the evolutionary dynamics.

## 1. Introduction

Researchers address evolutionary questions with various methods ranging from mathematical models to wet-laboratory and field experiments. These approaches are highly successful, but have limitations. For example, mathematical models make simplifying assumptions about complex ecological interactions in order to be tractable [1]. Long-term evolutionary experiments with organisms having generation times higher than bacteria are practically impossible [2]. Digital evolution performed in a computer has been advocated as an alternative and promising approach to bypass such limitations [2–8]. It operates on a finite population of individuals [9], each having a genome encoding its morphology and/or behaviour. The ‘selection method’ determines on the basis of individual performance which individuals will contribute offspring, after mutation and/or recombination, to the next generation.

Several selection methods are commonly used in digital evolution studies. The proportionate selection method (PSM) chooses the individuals contributing to the next generation proportionally to their performance scores [10]. The rank selection method (RSM) chooses a parent proportionally to the ranks (positions in a sequence of individuals sorted ascending by the performance scores) [11]. With both PSM and RSM, any individual has a chance to contribute to the next generation. By contrast, with the ‘truncation’ methods of selection, only a certain fraction of the population (i.e. the best performing individuals) contributes offspring to the next generation. The truncation-proportionate selection method (TPSM) chooses a parent proportionally to performance scores, whereas the truncation-uniform selection method (TUSM) chooses a parent uniformly at random [12–16]. Finally, the tournament selection method (TSM) forms ‘tournaments’ by sampling individuals with replacement uniformly at random from the entire population. The genotypes of the individuals with the highest performance scores in each tournament are copied to the descending generation [17,18].

While all five selection methods are frequently used to simulate differential selection (PSM in [19–32]; RSM in [33,34]; TPSM in [35–37]; TUSM in [38–46], TSM in [22,47,48]), the choice between them is rarely justified. Moreover, little attempt has been made to quantify the effects of selection methods on the dynamics of the digital evolution (but see [22,49]). This is a major issue, because each selection method defines a different mapping between performance scores and fitness (see the electronic supplementary material and figure 1), thus having important implications on the course of evolution [50–53]. To address this problem, we investigate theoretically and with numerical experiments how the five selection methods regulate the evolution of cooperation. We focus on cooperation, because digital evolution is especially popular in this domain [19–24,26–29,33,38,41,47,48,54,55], and it is an important biological phenomenon that has attracted extensive scientific interest (see [56–60] for reviews). We consider a population of related individuals, each having a genotype that consists of a haploid allele encoding for cooperation or defection. The individuals engage in a social game of Prisoner's Dilemma [61,62] where a cooperator pays a cost and its partner receives a benefit. In mathematical models, the cost and benefit of cooperation directly affect the fitness of the individuals [63,64]. We extend this approach by considering that the cost and benefit of cooperation affect performance scores, which are translated into fitness by one of the five selection methods (PSM, RSM, TPSM, TUSM and TSM). For each selection method, we identify with formal analysis the conditions in which cooperation evolves and we experimentally quantify its level.

## 2. Material and methods

### 2.1. Cooperation scenario

Individuals met each other and received performance scores equal to the pay-offs of the normalized linear Prisoner's Dilemma game [61] (table 1*a*). A cooperator received performance score *B* if it met a cooperator, and 0 if it met a defector. A defector received performance score *B* + *C* if it met a cooperator, and *C* if it met a defector. In the formal analysis, we also considered a more general version of the Prisoner's Dilemma game (see the electronic supplementary material). In the numerical experiments, we fixed the performance cost of cooperation *C* to 1.

The meetings of individuals were assortative, i.e. with probability *r*, an individual met an individual of the same strategy, and otherwise it met an individual chosen uniformly at random from the entire population [62] (table 1*b*). Thus, *r* measured the relatedness between individuals [65], i.e. it reflected the ‘surplus’ of probability that the social partners used the same strategy than was expected by random.

Prisoner's Dilemma game has been used to model the logic of animal conflict and cooperation [61]. One example is public goods production in bacteria where producers pay an energetic cost to secrete an enzyme and non-producers receive a benefit by freely absorbing it (see [66] for more examples). In this context, the positive relatedness would be due to spatial structure of the population where an individual's performance is affected only by neighbouring production and/or absorption of public goods. We further elaborate on the meaning of assortative meetings and relatedness in the Discussion.

### 2.2. Selection methods

With PSM, the probability of selecting the individual *i* is equal to , where *f _{i}* is the performance score of the individual

*i*. With RSM, the probability of selecting the individual

*i*is equal to , where

*r*is the rank of the individual

_{i}*i*, i.e. its position in the sequence of all individuals sorted ascending by performance scores (individuals with the same performance score had the same rank). With TPSM and TUSM, only the

*t*fraction of individuals with the highest performance scores in the population was considered viable. Let

*S*denote the set of viable individuals’ indices. With TPSM, the probability of selecting the individual

_{t}*i*is equal to if , and 0 otherwise. With TUSM, the probability of selecting the individual

*i*is equal to if , and 0 otherwise. Finally, TSM with the tournament size

*s*sampled with replacement

*s*individuals uniformly at random from the entire population, and selected the individual with the highest performance score among the

*s*individuals (ties were resolved uniformly at random).

Selection methods have many biologically relevant interpretations. For example, truncation selections may capture selection by predation when individuals meeting certain criteria can escape predators, survive and reproduce, whereas all others die with no offspring [12]. In tournament selection, individuals fight with each other for reproduction, a common situation in sexual selection [67]. We further elaborate on the biological meaning of selection methods in the Discussion.

### 2.3. Numerical experiments

We evolved a population of 1000 individuals in 11 treatments. In each treatment, we used a different selection method: PSM, RSM, TPSM with truncation threshold *t* = 0.8, 0.5, 0.2, TUSM with truncation threshold *t* = 0.8, 0.5, 0.2 and TSM with tournament size *s* = 2, 3, 5 (see figure 1 for illustration of the performance to fitness mapping defined by each selection method). For each treatment, we investigated 40 × 51 conditions, with the performance cost of cooperation *C* fixed to 1, the performance benefits of cooperation *B* ranging between 1.1 and 5 with a step of 0.1, and the relatedness level *r* ranging between 0 and 1 with a step of 0.02. For each treatment and each condition, we replicated the numerical experiment 30 times. The values of *B* and *r* are kept constant during an evolutionary run.

### 2.4. Genetic architecture, selection and reproduction

Each individual had a genotype consisting of one binary allele denoting the lack (0) or the possession (1) of the cooperative trait. We, thus, assumed a one-to-one mapping from genotype to phenotype. We used an on–off transition between cooperation and defection to get clearer results on the processes regulating the evolution of cooperation under different methods of selection. Such a strong on–off transition may also occur in natural situations. For example, an ant worker contributes to the queen's wellbeing and forgoes reproduction, or lays its own eggs instead [68]. Similarly, a single cell secretes an enzyme and pays energetic cost, or saves energy and rides on public goods [69].

At the beginning of each evolutionary run, all 1000 individuals had the allele set to 0. At each generation, every individual was evaluated in the cooperation scenario and received a performance score. To construct the descending generation, 1000 individuals were independently selected from the entire population by a selection method that depended on the treatment. These 1000 individuals were mutated (with probability 0.001 the value of an allele was flipped) and then replaced all 1000 individuals from the population. Each evolutionary run lasted for 1000 generations.

In the experiments presented in the main text, we assumed that all individuals in the population are replaced every generation. We also performed additional experiments in which we investigated the effects of the generational overlap by using the Moran process [70] to update the population (see the electronic supplementary material).

### 2.5. Statistical analysis

For each replicate, treatment and condition, we measured the evolved cooperation level as the proportion of cooperators in a population averaged across generations 900–1000. To compare the selection methods, we considered mean cooperation level over all conditions (figure 2). To investigate the effect of the relatedness level *r*, we quantified the average cooperation level over all conditions with the same value of *r* (figure 3, solid line). To investigate the effect of the performance benefit of cooperation *B*, we quantified the average cooperation level over all conditions with the same value of *B* (figure 3, dashed line). Statistical significance between all treatments was determined with the Kruskal–Wallis test (non-parametric one-way analysis of variance) and between a pair of treatments with the Wilcoxon test (rank sum test for equal medians).

## 3. Results

### 3.1. Formal analyses

Prior to performing numerical experiments, we investigated with formal analysis the spread of a cooperative allele in an infinite population of individuals related at level *r* on average (see the electronic supplementary material). A cooperator paid a cost *C* > 0 which is subtracted from its performance score, whereas the partner of a cooperator received a benefit *B* > *C* which is added to its performance score. The performance scores of the individuals were translated into their fitness values by one of the five selection methods: PSM, RSM, TPSM, TUSM and TSM. Two of them (PSM and RSM) do not have any free parameters. By contrast, TPSM and TUSM are characterized by the truncation threshold *t*, which determines the fraction of individuals (i.e. those with the highest performance scores) which are viable. Finally, TSM is characterized by the tournament size *s*, which indicates the number of individuals that compete between each other in randomly formed groups. Using each of the five selection methods, we identified the conditions for evolution of cooperation (table 2*a*) and defection (table 2*b*).

In general, we found that the conditions for evolution of cooperation and defection depend on the frequency of cooperators in a population (see the electronic supplementary material). Thus, we focused on the conditions when the invading allele is under positive selection, assuming that the opposite allele has reached fixation. Consequently, one can predict when either the cooperative or the defective allele reaches fixation, and when the two alleles coexist in the population (table 2). In particular, cooperation reaches fixation if it is under positive selection in a population of defectors, and defection is not under positive selection in a population of cooperators. Similarly, defection reaches fixation if it is under positive selection in a population of cooperators, and cooperation is not under positive selection in a population of cooperators. Finally, the population is polymorphic if both cooperation is under positive selection in a population of defectors, and defection is under positive selection in a population of cooperators.

With PSM, cooperation is under positive selection in a population of defectors when *r* > *C*/*B*, whereas defection is under positive selection in a population of cooperators when *r* < *C*/*B*. With TPSM, these conditions are relaxed, and cooperation is under positive selection in a population of defectors when *r* > *tC*/*B*, whereas defection is under positive selection in a population of cooperators when *r* < 1 – *tB*/(*B* + *C*). In contrast to PSM and TPSM, with RSM, TUSM and TSM, the conditions for invasion of cooperation and of defection are independent of *C* and *B*. Cooperation is under positive selection in a population of defectors when *r* > 1/2 for RSM, *r* > *t* for TUSM, and *r* > 1/*s* for TSM. Similarly, the defection is under positive selection in a population of cooperators when *r* < 1/2 for RSM, *r* < 1 – *t* for TUSM, and *r* < 1 – 1/*s* for TSM. All results are jointly presented in table 2 (see the electronic supplementary material for formal derivations and extended analyses).

### 3.2. Numerical experiments

To verify and extend our formal analyses, we performed numerical experiments and quantified the average cooperation level in 30 independent populations under each of the five selection methods. Overall, there were 11 treatments, because we systematically investigated different values of the truncation threshold *t* and of the tournament size *s* (PSM, RSM, TPSM *t* = 0.8, 0.5, 0.2, TUSM *t* = 0.8, 0.5, 0.2, TSM *s* = 2, 3, 5). For the sake of simplicity, we fixed the performance cost of cooperation *C* = 1, and investigated the combined effects of the relatedness level *r* and the performance benefit of cooperation *B* on the cooperation level. There were significant differences in the level of cooperation averaged across all conditions between all 11 treatments (figure 2, Kruskal–Wallis test, d.f. = 10, *p* < 0.001) and between each pair of treatments (55 pairwise Wilcoxon tests, d.f. = 29, all *p* < 0.001).

The performance benefit of cooperation *B* had different effects on the level of cooperation depending on the selection method used. The cooperation level increased with *B* in the four treatments with PSM and TPSM *t* = 0.8, 0.5, 0.2. By contrast, in the five other treatments, the value of *B* had no effect on the level of cooperation (figure 3, dashed line).

In all 11 treatments, the level of cooperation increased with relatedness *r* (figure 3, solid line). However, there were differences among treatments about the nature of the transition from defection to cooperation. There was a thresholding effect in the four treatments with RSM, TUSM *t* = 0.8, TUSM *t* = 0.5 and TSM *s* = 2 as the evolved populations contained either defectors (for low values of *r*) or cooperators (for high values of *r*). By contrast, in the seven other treatments, the transition from defection to cooperation with the increase of *r* was gradual. This was because the cooperation level depended on combined effects of *r* and *B* (figure 4*a*, PSM, TPSM *t* = 0.8, 0.5, 0.2), and because polymorphic populations evolved (figure 4*a*, TPSM *t* = 0.8, 0.5, 0.2, TUSM *t* = 0.2, TSM *s* = 3, 5).

For all treatments, the outcomes of the numerical experiments were in good agreement with the predicted conditions where cooperation should invade defection, and vice versa (figure 4*a,b*). Cooperation level was low in conditions where a population of defectors was predicted to be resistant against the invasion of cooperation (figure 4*a*, area below the solid line). By contrast, cooperation always went to fixation in conditions where it was predicted to invade a population of defectors and defection was not predicted to invade a population of cooperators (figure 4*a*, area above the dashed line). Finally, populations were polymorphic in conditions for which both cooperation was predicted to invade a population of defectors and defection was predicted to invade a population of cooperators (figure 4*a*, TPSM *t* = 0.8, 0.5, 0.2, TUSM *t* = 0.8, 0.5, 0.2 and TSM *s* = 3, 5, intersection of the area below the dashed line and the area above the solid line).

Despite the good agreement between formal analyses and experimental results, there were few small discrepancies. In contrast to the predicted conditions for evolution of cooperation under TUSM (i.e. *r* > *t*), cooperation evolved when *r* was slightly lower than *t* (i.e. *r* = 0.78 instead of 0.8 with TUSM *t* = 0.8, and *r* = 0.46, 0.48 instead of 0.5 with TUSM *t* = 0.5). Similarly, in contrast to the predicted conditions for evolution of cooperation under TSM (i.e. *r* > 1/*s*), cooperation evolved in conditions when *r* was slightly lower than 1/*s* (*r* = 0.48 instead of 0.5 with TSM *s* = 2). These small discrepancies stem from the effects of mutation in finite populations, which relaxed the conditions for evolution of cooperation with TUSM and TSM (see the electronic supplementary material).

In all 11 treatments and in all conditions, the evolved populations were stable. The only exception was under TPSM with *t* = 0.2 and TUSM with *t* = 0.2 in conditions with *r* ranging between 0.25 and 0.5 (see the electronic supplementary material). Owing to the low value of the truncation threshold in these two methods of selection, the number of parents was small in relation to the population size. Low effective population size has been shown to suppress selection, and amplify random drift [12,71], which explains the instability of the evolved populations. Interestingly, a high level of relatedness (*r* > 0.5) prevented instability of populations.

Additionally, instead of replacing the entire population of 1000 individuals at each of the 1000 generations, we used the Moran process [70] to replace one individual at each of the 1000 × 1000 iterations. We found very good agreement between the cooperation levels that evolved without and with the generational overlap, for all treatments under all investigated conditions (compare the electronic supplementary material, figure S2 and figure 4*b*).

## 4. Discussion

The results show that cooperation level is positively associated with the relatedness level, regardless of the selection method used. This supports the long recognized view [28,52,72] that relatedness between individuals (or, in general, any kind of positive assortment between individuals of the same type [62,73]) facilitates the evolution of cooperation. However, the results also show that selection methods regulate the evolution of cooperation, by strengthening or relaxing the conditions in which cooperation could evolve. For example, a significant level of cooperation evolved in a wide range of conditions under TPSM and TUSM with a low truncation threshold (*t* = 0.2), and under TSM with high tournament size (*s* = 5). Because low truncation threshold and high tournament size, both reflect high selection pressure, this suggests that cooperation may originate more easily (i.e. with lower relatedness) in harsh environments, or at the beginning of invasion events when the population is not yet well adapted to the new environment.

The level of cooperation was also positively associated with the benefits of cooperation on performance, but only under PSM and TPSM. With the other selection methods, the value of the performance benefit did not affect the cooperation level. This is because, RSM, TUSM and TSM select the genomes of the individuals to the descending generation based only on the ranked performance scores (i.e. whether one individual has a higher performance score than the other). Thus, the effect of cooperation on performance scores does not affect the likelihood of individuals to contribute to the next generation under RSM, TUSM and TSM, provided that the general relationship *B* > *C* > 0 holds. Consequently, with non-PSMs all Prisoner's Dilemma games are equivalent for a given relatedness level, and lead to the same evolutionary outcome. This result is consistent with a previous report on a hawk–dove game played in populations of unrelated individuals [49].

Our analyses also demonstrate that polymorphic populations evolve with TPSM, TUSM and TSM in some conditions. In polymorphic populations, cooperation and defection coexist simultaneously which is a sign of frequency-dependent selection. The reproductive advantage of cooperators over defectors depends on their proportion in the population. With a low proportion of cooperators, they have the reproductive advantage and increase in numbers. However, with a high proportion of cooperators, they lose the reproductive advantage and decrease in numbers. Overall, TPSM, TUSM and TSM act in such conditions as balancing selection that stabilizes the cooperation level at an intermediate value. Similar conclusions were reached for cooperation evolving under selection in ephemeral networks [74], which in fact resemble the tournament selection method.

At this point, we should discuss the relatedness level *r* in a broader context. Here, *r* was explicitly defined as the ‘surplus’ of probability that a focal individual meets an individual which is using the same strategy (in comparison with random expectation). The relatedness level can also be expressed as where *x _{i}* is the strategy of the

*i*th individual,

*y*is the strategy of the social partner of the

_{i}*i*th individual and

*i*enumerates over all individuals in the population. Thus, in general,

*r*measures any kind of assortment between individuals of the same type relative to the population's average, an idea already put forward by Hamilton [73], and formalized by Queller [75]. Intuitively, the relatedness level reflects the amount of available information about the social partner that the evolution can take advantage of [76]. Consequently, our results extrapolate beyond the situations in which relatedness is strictly due to assortative meetings.

To begin with, *r* might reflect identity by descent [28], i.e. an assortative meeting of level *r* is as if the two social partners shared a recent common ancestor with probability *r* (assuming that, like in this paper, individuals are haploid). Consequently, our *r* is equivalent to the haploid version of the relationship coefficient that measures the average fraction of genes identical by descent, as used by Hamilton in the landmark paper on inclusive fitness [72]. Similarly, the simple genetic structure of our population makes the relatedness level *r* equivalent to Wright's *F*-statistics, i.e. *F* = 1 − *o*/*e* = *r,* where *o* = 2*p*(1 – *r*)(1 – *p*) is the fraction of individuals that met an individual of an opposite strategy under assortative meetings, and *e* = 2*p*(1 – *p*) is the fraction of individuals that would have met an individual of an opposite strategy under completely random meetings with no assortment (*p* is the proportion of cooperators in a population).

So far, we have described the relatedness *r* as a population's statistic, but mechanistic interpretations pointing to proximate causes of positive assortment are also possible. Then, *r* might be interpreted in terms of learning by imitation where an individual mimics the behaviour of its social partner with probability *r* [48]. Alternatively, *r* might be linked to migration in viscous populations, where with probability *r* an individual does not migrate (and, thus, meets an individual of the same type) and with probability 1 – *r* it migrates and, thus, meets a randomly chosen individual [77].

Despite these various interpretations of positive relatedness, we must point out two limitations of our study. First, we did not directly study the proximate causes of assortative meetings, as our model dictates that they simply do happen with the given probability *r*. It has been shown that different mechanisms such as kin recognition, viscous populations and green-beard genes facilitate the evolution of cooperation to different degrees [78]. How these intrinsic differences between the mechanisms interplay with the selection methods studied in this paper remains an interesting avenue for future research. Second, we did not consider the ultimate causes of assortative meetings, as we did not evolve the relatedness level which was fixed across generations. The question of how the relatedness co-evolves with cooperation [77,79] is important and to a large degree still open, but was outside the scope of this paper.

The remaining question is which method of selection to use in digital evolution. The short answer is that the choice depends on the purpose of the model. With PSM, the performance score is equal to fitness, because fitness is linearly proportional to performance scores (see the electronic supplementary material) and because fitness is irrelevant to scaling [80]. Thus, digital evolution using PSM directly corresponds to mathematical models which usually simply use fitness values [63].

Truncation selection was proposed based on the observation that many biological processes display a thresholding effect [12,14,13]. For example, only the rabbits with a running speed higher than a certain threshold value may escape predators, survive and reproduce [12]. Fitness distributions supporting the idea of truncation were observed in social insects in the wild [81]. Overall, truncation selection methods approximate natural selection by predation [82], when the weakest are eliminated from the gene pool. They also resemble to some extent a purifying selection, which removes deleterious mutations [83]. From yet another perspective, selection with truncation mimics competition for limited resources in highly mobile species, such as the competition for nest sites in birds.

Finally TSM, which simulates direct competition in small groups, resembles intrasexual selection where individuals of the same sex directly compete between each other to reproduce, and unsuccessful competitors have few or no offspring [67]. This method is also similar to selection in ephemeral networks (i.e. short-lasting groups in which individuals interact and compete, which form in microbes, marine invertebrates, annual plants and other organisms; see [74] for more details).

The measurement of fitness of organisms in the wild is difficult, and there has been considerable discussion about how to measure fitness in natural populations and how to represent it mathematically [80]. Our formal analyses and numerical experiments indeed show that each of the five commonly used selection methods regulates the evolution of cooperation in a distinct way. The difference in outcomes between the selection methods stems from differences in the mapping between performance and the relative contribution of genotypes to the next generation. The actual mapping between phenotype and fitness poses a great challenge for both evolutionists and ecologists, because it likely depends on many factors such as the nature of intra- and interspecific competition [84]. Consequently, the choice of a selection method, although often marginalized, is a crucial step in the modelling process as it has important implications on the evolutionary outcome of the investigated traits.

## Funding statement

This work was supported by the Swiss National Science Foundation and an ERC advanced grant.

## Acknowledgements

We thank Barbara Piasecka, Steffen Wischmann, Laurent Lehmann, Giovanni Iacca and the two anonymous reviewers for useful comments on the manuscript.

- Received August 12, 2013.
- Accepted October 2, 2013.

- © 2013 The Author(s) Published by the Royal Society. All rights reserved.