Abstract
Rock is wrapped by paper, paper is cut by scissors and scissors are crushed by rock. This simple game is popular among children and adults to decide on trivial disputes that have no obvious winner, but cyclic dominance is also at the heart of predator–prey interactions, the mating strategy of sideblotched lizards, the overgrowth of marine sessile organisms and competition in microbial populations. Cyclical interactions also emerge spontaneously in evolutionary games entailing volunteering, reward, punishment, and in fact are common when the competing strategies are three or more, regardless of the particularities of the game. Here, we review recent advances on the rock–paper–scissors (RPS) and related evolutionary games, focusing, in particular, on pattern formation, the impact of mobility and the spontaneous emergence of cyclic dominance. We also review meanfield and zerodimensional RPS models and the application of the complex Ginzburg–Landau equation, and we highlight the importance and usefulness of statistical physics for the successful study of largescale ecological systems. Directions for future research, related, for example, to dynamical effects of coevolutionary rules and invasion reversals owing to multipoint interactions, are also outlined.
1. Introduction
Games of cyclic dominance play a prominent role in explaining the intriguing biodiversity in nature [1–3], and they are also able to provide insights into Darwinian selection [4], as well as into structural complexity [5] and prebiotic evolution [6]. Cyclical interactions have been observed in different ecological systems. Examples include marine benthic systems [7], plant systems [8–12] and microbial populations [2,13–16]. Cyclic dominance also plays an important role in the mating strategy of sideblotched lizards [17], the overgrowth of marine sessile organisms [18], the genetic regulation in the repressilator [19], and in explaining the oscillating frequency of lemmings [20] and of the Pacific salmon [21]. The list of examples where the puzzle of biological diversity can be explained by the interaction topology of the food web among those who struggle for life is, indeed, impressively long and inspiring [22,23].
In addition to the closed loops of dominance in the food webs that govern the evolution of competing species and strategies, experiments have shown that the key to the sustenance of biodiversity is often spatial structure. In vitro experiments with Escherichia coli, for example, have shown that arranging the bacteria on a Petri dish is crucial for keeping all three competing strands alive [2,24,25]. On the other hand, in vivo experiments with bacterial colonies in the intestines of cocaged mice, which can be considered as locally wellmixed populations, have revealed that mobility alone is sufficient to maintain coexistence [14]. Solely because the bacteria were able to migrate from one mouse to another, the biodiversity was preserved. Motivated by such intriguing experimental observations, theoretical explorations of the relevance of spatial structure in evolutionary games have also received ample attention in the recent past [26–31].
The importance of spatial structure has been put into the spotlight by the discovery that interaction networks such as the square lattice can promote the evolution of cooperation in the Prisoner's Dilemma game through the mechanism that is now known as network reciprocity [32,33]. In essence, network reciprocity relies on the fact that cooperators do best if they are surrounded by other cooperators. More generally, the study of spatial games has revealed the need for realistic models to go beyond wellmixed populations [34–36]. To understand the fascinating diversity in nature and to reveal the many hidden mechanisms that sustain it, we cannot avoid the complexity of such systems. Cyclical interactions are characterized by the presence of strong fluctuations, unpredictable and nonlinear dynamics, multiple scales of space and time and frequently also some form of emergent structure. Because many of these properties are also inherent to problems of nonequilibrium statistical physics, theoretical research on spatial rock–paper–scissors (RPS) and related games of cyclic dominance has a long and fruitful history [37–65], much of which is firmly rooted in methods of statistical physics that can inform relevantly on the outcome of evolutionary games on structured populations.
Cyclical interactions are in many ways the culmination of evolutionary complexity, and they may also arise spontaneously in evolutionary games where the number of competing strategies is three or more. For example, cyclic dominance has been observed in public goods games with volunteering [66,67], peer punishment [35,66,68,69], pool punishment [70,71] and reward [72–74], but also in pairwise social dilemmas with coevolution [75,76] or with jokers [77]. Counterintuitive complex phenomena that are due to cyclic dominance include the survival of the weakest [41,49], the emergence of labyrinthine clustering [64] and the segregation along interfaces that have internal structure [78], to name but a few examples. Importantly, the complexity of solutions increases further and fast if the competing strategies are more than three. Figure 1 depicts a transition towards more and more complex food webs from left to right, which support the formation of defensive alliances [81] and subsystem solutions [82] that are not possible in the simplest RPS game.
Although the outcome of evolutionary games with cyclic dominance may depend on the properties of the interaction network, the topology of the food web, and on the number of competing strategies, there, nevertheless, exists fascinatingly robust universal behaviour that is independent on model particularities. While we may have only begun to scratch the surface of the actual importance of cyclic dominance in nature [83,84], the time is ripe for a survey of theoretical advances that may guide future experimental efforts. Like by the evolution of cooperation [85], in the games of cyclic dominance, the evolutionary game theory [86–90] is also proving to be the theory of choice for the study of these beautifully simple yet fascinating systems.
The organization of this review is as follows. In §2, we focus on meanfield and zerodimensional RPS models to give a comprehensive prologue to the overview of the spatial RPS game in §3. Within the latter, we separately review the impact of different interaction networks and mobility, and we describe the derivation and the proper use of the complex Ginzburg–Landau equation (CGLE) to describe the spatiotemporal properties and the stability of the spiralling patterns. In §4, we depart from the RPS game where the cyclic dominance is explicitly encoded in the food web by considering social dilemmas and other evolutionary games. We separately review games where cyclical interactions emerge spontaneously, the role of timedependent interactions, the impact of voluntary participation, and we survey evolutionary games where an alliance between two strategies acts as an additional independent strategy. From there, we focus on cyclical interaction between four and more competing strategies, which opens the door towards reviewing the emergence of defensive alliances and large networks of competitive species. We conclude with a summary and an outlook towards promising future research efforts.
2. Cyclic dominance in wellmixed systems
While the emphasis of this review is on cyclic dominance in structured populations, it is useful to start by considering the simplest setting to study RPS games which is the mean field limit of wellmixed populations where all individuals can interact with each other.
A generic RPS game between individuals of three species, A, B and C, in a population of total size N entails cyclic competition and reproduction according to the following reaction schemes 2.1 2.2 2.3 where the reactions (2.1) and (2.2) describe two forms of cyclic dominance of species A over B, B over C and in turn, C over A, and (2.3) accounts for reproduction into an empty space (denoted by ∅︀) with a rate q [28,46,91]. The reaction given by equation (2.1) corresponds to one species dominating and displacing another (dominanceremoval) with rate p, whereas, in equation (2.2), there is a zerosum dominancereplacement with rate z. Recent experiments have shown that in addition to cyclic dominance, some species can mutate. For instance, E. coli bacteria are known to mutate [2], and it was found that the sideblotched lizards Uta stansburiana are not only engaged in cyclic competition, but they can also undergo throat colour transformations [92]. It is therefore also instructive to assume that each species can mutate into one of the other species with a rate μ [53,91], according to 2.4 Additional evolutionary processes that should be specified are those encoding the movement of individuals. However, these are irrelevant in the absence of spatial structure and is therefore introduced in §3. In the mean field limit of population size N → ∞, the dynamics of the system evolving according to equations (2.1)–(2.4) is aptly described by the following rate equations for the densities a(t) of As, b(t) of Bs and c(t) of Cs 2.5 where ρ_{∅︀} = 1 − (a + b + c) denotes the density of empty sites. These rate equations admit a steadystate s* = (a*, b*, c*) in which A, B and C coexist at the same density a* = b* = c* = q/(p + 3q) encompass three types of oscillatory dynamics:

When p, q > 0, z ≥ 0 and μ = 0, the equation (2.5) can be rewritten as the rate equations of the May–Leonard model [93]. In this case, the above coexistence state is unstable, and there are also three absorbing stationary states corresponding to the population being composed of only A, B or C. As in the May–Leonard model, the dynamics is characterized by heteroclinic cycles connecting these three absorbing states, with the population being composed almost exclusively of either A, B or C, in turn. It is worth noting that the heteroclinic cycles are degenerate when z = 0 [93].

When p, q > 0, z ≥ 0 and μ > 0, a supercritical Hopf bifurcation arises at the value μ = μ_{H} = pq/(6(p + 3q)) of the bifurcation parameter: the coexistence steadystate is a stable focus when μ > μ_{H}, whereas it is unstable when μ < μ_{H}, and the dynamics is thus characterized by a stable limit cycle of frequency [53,91].

When z > 0 and p = q = μ = 0, the state ∅︀ plays no role in the dynamics, and one recovers the cyclic Lotka–Volterra model [39,94]. The sum and product of the densities, a(t) + b(t) + c(t) and a(t)b(t)c(t), are conserved by (2.5) which results in nested neutrally stable closed orbits around the coexistence fixed point which is a neutrally stable centre [95]. It is worth noting that the cyclic dominancereplacement scheme with asymmetric rates is also characterized by a marginally stable coexistence fixed point s* = [1/(z_{A} + z_{B} + z_{C})](z_{B}, z_{C}, z_{A}) surrounded by neutrally stable orbits on which the mean field dynamics takes place (figure 2) [42,50].
The evolution in wellmixed populations of finite size, N < ∞, is usually described in terms of birth–death Markov chains [96]. In this case, the interactions between a finite number of discrete individuals lead to fluctuations that may drastically affect the dynamics. In particular, in the presence of absorbing states, fluctuations are responsible for the extinction of two strategies and the fixation of the surviving one (figure 2) [42,50]. Questions concerning the mean time necessary for two species to go extinct and the probability that one specific strategy survives have recently attracted significant attention and were mostly addressed by means of stochastic simulations. These are often efficiently carried out using the Gillespie algorithm [97]: for instance, cyclic competition and reproduction according to equations (2.1)–(2.3) are implemented by allowing at each time increment a reproduction event with probability q/(p + q + z), a dominance displacement move with probability p/(p + q + z) and a dominancereplacement update with probability z/(p + q + z). When p > 0 and μ = 0 and the mean field dynamics is characterized by heteroclinic cycles, the mean extinction time has been found to be of order . The existence of quantities conserved by the mean field dynamics allows some analytical progress in the case of cyclic dominance with replacement (p = q = μ = 0) for which it was shown that two species die out after a mean time that scales linearly with the system size N [42,98,99]. In this class of models with asymmetric rates, it was also shown the survival probability follows an intriguing ‘law of the weakest’: the strategy with the smallest dominance rate is the most likely to survive and to fixate the population [41,49], as illustrated in figure 2. Some other aspects of extinction in RPS games in finite and wellmixed populations have been considered in references [100,101] and studied in further details in [102].
3. Rock–paper–scissors game in structured populations
In a structured population, or alternatively, in a spatial system, the interactions between players are defined by an interaction network and are thus limited. Because players have only limited interactions with their neighbours, a strategy (or species) will not necessarily meet its superior (or predator). This restriction has farreaching consequences because it enables pattern formation, and through that the survival of all three competing strategies with a timedependent frequency akin to the red queen [103–105]. For example, it is possible to observe propagating fronts and spiral waves between the competing strategies [40,47,91], which are particularly fascinating and complex if the invasion rates between different pairs of strategies are significantly different [64]. In general, selfsustained oscillations of all three strategies are possible in spatially homogeneous populations [106,107] and result from periodic dynamics around a longlived metastable coexistence state whose lifetime is determined by the system size (or the size of the population) [108,109]. Hence, if the system size is sufficiently large all possible combinations of invasion fronts appear, resulting in a timeindependent constant frequency of strategies, depending only on the invasion rates between the three competing strategies.
In comparison with a wellmixed population, a spatial system allows us to introduce an additional microscopic process that has no effect if everybody interacts with everybody else, namely site exchange between strategies or between an individual player and an empty site. This gives rise to mobility, which arguably has a profound impact on the outcome of the RPS game as it can both promote and impede biodiversity [3]. In contrast to wellmixed systems, where as seen intrinsic noise can jeopardize the coexistence of species [42], depending on the level of mobility, species coexistence can be more easily maintained than under wellmixed conditions [3,45], and stochastic effects can have counterintuitive effects at some bifurcation point be reduced by stable patterns in spatial systems [46].
The mentioned mixing of strategies (directly or via empty sites) can also influence the cyclic competition in aquatic media [110]. Furthermore, the degree of mixing determines the way of extinction, such as the annihilation of domains, heteroclinic orbits or travelling waves in a finite system [111]. The RPS game can also be regarded as a cyclical Lotka–Volterra model, and indeed, it has been shown that a twospecies Lotka–Volterra system with empty cites can give rise to global oscillations [112]. In general, however, it is important to acknowledge that the structure of the interaction network and mobility can both have a significant impact on the evolutionary outcome of cyclical interactions, and in the following subsections, we review this in more detail as well as survey the usefulness of the CGLE.
3.1. Interaction networks
In a structured population, species should contact in realspace and their interaction is characterized by an appropriate graph. Lattices are the simplest among these interaction networks, where every player has the same number of neighbours. By far the most widely used is the square lattice, although alternatives such as the honeycomb lattice, the triangular lattice and the kagome lattice are frequently employed as well to study the relevance of the degree of players and the role of clustering. While the square and the honeycomb lattice both have the clustering coefficient equal to zero, the kagome and the triangular lattice both feature percolating overlapping triangles, such that their clustering coefficient is and , respectively. In general, lattices can be regarded as an even field for all competing strategies where the possibility of network reciprocity is given [32]. Furthermore, as there are many different types of lattices, it is possible to focus on very specific properties of cyclical interactions and test what their role is in the evolutionary process. In some cases, such as competition between bacteria [2,16,28,113,114] or for the description of parasite–grass–forb interactions [12], lattices are also an apt approximation of the actual interaction network. In more complex systems, however, the description of interactions requires the usage of more intricate networks, which typically have broadscale degree distribution and smallworld properties [115,116]. The question how the topology of the interaction network affects the outcome of cyclical interaction, and gametheoretical models, in general, [26,27,29,31], is one of great interest.
As an initial departure from lattices towards more complex interaction topologies, it is possible to consider networks with dispersed degrees. As demonstrated by Masuda and Konno, the resulting heterogeneity can help to maintain the stable coexistence of species in cyclic competition [117]. Another possibility is to introduce permanent shortcut links that connect distant players, thereby effectively introducing smallworld properties. In contrast to lattices, on smallworld networks, the frequencies of competing strategies oscillate even in the large system size limit, as shown in [118–121]. The explanation behind this observation lies in the fact that the introduction of shortcuts introduces quenched randomness into the interaction network. If the magnitude of this randomness exceeds a threshold value, i.e. if the number of shortcut links is sufficiently large, the amplitude of frequency oscillations can be so large that the system will always terminate into an absorbing state during the evolutionary process. These results lead to the conclusion that not only the limited interaction range that is imposed by structured populations, but even more so the explicit spatial distribution of competing species is the crucial property that determines coexistence and biodiversity [122]. This observation becomes particularly interesting if we compare it with the observations made in the realm of spatial social dilemma games, where the limited interaction range was found to be the decisive factor for the promotion of cooperation based on network reciprocity [26,33].
Properties that are typical of small worlds can be introduced not only by means of permanent shortcut links between distant players, but also by means of temporary longrange links that replace nearestneighbour links with probability P. This process is typically referred to as annealed randomness [118]. But while topological properties of annealed and quenched small worlds may be very similar, there is a relevant difference in the evolutionary outcomes of cyclical interactions taking place on them. Namely annealed randomness no longer gives rise to oscillatory frequencies of the three competing strategies in the RPS game. For sufficiently large values of P, the oscillations are replaced by steadystate behaviour [123], as is commonly observed on regular lattices. Interestingly, similar results were also reported for a fourstrategy cyclic dominance game in [124]. But as we review later, the transitions between stationary and oscillatory states in cyclical interactions can, in fact, be driven by many different mechanisms, even if the interaction network thereby remains unchanged.
The topology of the interaction network may also change as a result of a particular coevolutionary rule, as studied frequently before in the realm of social dilemma games. Note that when not only individual strategy, but also other playerspecific character (such as connectivity, strategy adopting or passing capacity, etc.) may change in time then we can observe parallel evolutions, coevolution of strategies and other quantities, see [29] for a review. In particular, rewiring could be the consequence of invasions between strategies [75,125]. Although games of cyclic dominance received comparatively little attention in this respect, there is a study by Demirel et al. [126], which builds exactly on this premise. Namely the loser of each particular instance of the game can either adopt the winning strategy (an invasion occurs), or it can rewire the link that brought the defeat. It was shown that nonequilibrium phase transitions occur as a function of the rewiring strength, as illustrated in figure 3. The observation of the ‘stationary → oscillatory → absorbing → stationary’ phase transitions is conceptually similar to those observed in the realm of interaction networks with annealed randomness. In general, it is thus possible to conclude that oscillatory behaviour in coevolving networks is a robust phenomenon—a conclusion that is further corroborated by agentbased simulations showing that coevolving networks can influence the cycling of host sociality and pathogen virulence [127].
3.2. Mobility
As briefly explained above, besides the properties of the interaction network, mobility, as is the case in general for evolutionary gametheoretical models [128–132], can also play a decisive role in the maintenance of biodiversity in games of cyclic dominance. The consideration of mobility has both theoretical and practical aspects. Theoretically, the question is whether mobility affects the stability of spatial patterns that typically emerge in structured populations, and if yes, how? From a practical point of view, it is important to acknowledge the fact that in reallife prey and predators are frequently on the move in order to maximize their chances of survival. Accordingly, their neighbourhoods change, which is in agreement with the effect that is brought about by mobility in simulations. The crucial point to clarify is whether mobility simply shifts the evolutionary outcomes towards those observed in wellmixed populations, or whether there emerge unexpected phenomena that disagree with meanfield predictions. As we review in what follows, the introduction of mobility in structured populations is certainly much more than simply a transition towards wellmixed conditions.
Pioneering work on the subject has been done by Reichenbach et al. [3,133], who showed that mobility (or migration) has a critical impact on biodiversity. In particular, small mobility that is below a critical threshold promotes species coexistence, whereas large mobility that is above the threshold destroys biodiversity. Behind these observations are spontaneously emerging spiral patterns, which, in general, sustain the coexistence of all three competing strategies [40,134]. Under the influence of mobility, however, the spirals grow in size, and above the critical threshold, they outgrow the system size causing the loss of biodiversity. Accordingly, in [3], it was thus argued that mobility can both promote and jeopardize biodiversity in RPS games.
As most studies on this subject, in this section, we focus on RPS games on twodimensional square lattices of total size N (here, linear size is ). The population's composition changes according to the dominanceremoval reactions given by equation (2.1) and reproduction reactions given by equation (2.3) (i.e. q, p > 0 and z = μ = 0, which yields degenerate heteroclinic cycles at mean field level; see §2). Furthermore, it is assumed that individuals can move by swapping their position with any neighbour at rate γ according to the scheme 3.1 where . In many references, mobility is measured in continuum limit by introducing the mobility or diffusion coefficient m = 2γ/N, where N is the system size [3].
To focus on the effects of mobility m on pattern formation and biodiversity, the rate of competition p and reproduction q can initially be set fixed at 1. Starting from random initial conditions where each lattice site is with equal probability either occupied by rock A, paper B, scissor C or left empty, we find that the emerging spatial patterns depend sensitively on mobility. As shown in the top row of figure 4, as mobility increases from left to right, the characteristic scale of spiral waves also increases. It was found that at a critical value of mobility, m = m_{c} = (4.5 ± 0.5) × 10^{−4} for p = q = 1, the length scale of the spiralling patterns outgrows the system size resulting in two species dying out and the dynamics settling in the absorbing of the remaining species [45–47,136]. The diversity is lost, whereby the extinction probability is defined as the disappearance of two species after a waiting time that is proportional to the system size. As the system size increases, this probability approaches zero if m < m_{c}, and it tends to 1 if m > m_{c}.
In contrast to mobility, competition promotes coexistence of species because the typical length of the spatial patterns decreases with increasing competition rate p. This is illustrated in the bottom row of figure 4 from left to right. Macroscopic spiral waves emerge at p = 0.1, but they become smaller and fragmented as p increases. The fragmentation is due to the fact that large competition rates introduce empty sites around small patches that contain all three competing strategies (i.e. at the spiral core), which, in turn, leads to the disintegration of macroscopic spirals typically observed away from the core. Diversity is therefore favoured at larger values of p, as they effectively prevent the spirals outgrowing the system size. Similar phenomenon can also be observed in the generic metapopulation model. In the latter case, however, the fragmentation is a consequence of the meeting of spiral fronts, as illustrated figure 8 [91,137].
Because pattern formation obviously plays a critical role in warranting the coexistence of the three competing strategies, it is particularly important to understand how the spatial patterns evolve. Prepared initial conditions are strongly recommended in such cases, as they can help identify critical processes that may either enhance or destroy the stability of coexistence. The emergence of spirals can be engineered if the evolution is initiated from three roundish areas, each holding a particular strategy, whereas the rest of the lattice is empty [138]. Based on such a procedure, it has been shown that there exists a critical competition rate, p_{c}, below which singlearmed spirals in finite populations are stable. For p > p_{c}, the spirals break up and form disordered spatial structures, because too many empty sites allow the emergence of competing mini spirals [135]. This transition is demonstrated in the top row of figure 5 from left to right.
Although Monte Carlo simulations are applied most frequently for studying spatial evolutionary games (see reviews [26] and [31] for details, and [91] for the use of the spatial Gillespie algorithm [97]), when density fluctuations are neglected, the dynamics of the spatial RPS game on a domain of unit size can be described by the following partial differential equations (PDEs) obtained by letting the lattice spacing 1/N vanish in the continuum limit N → ∞: 3.2 where r labels the twodimensional position, whereas ρ_{Ø} and D_{m} denote the density of empty site and the diffusion coefficient (on a domain of unit size, D_{m} coincides with m), respectively. This set of PDEs can be solved numerically by standard finite difference methods, and as illustrated in the bottom row of figure 5, the results are similar to those obtained via Monte Carlo simulations.
By exploiting further the potential of prepared initial conditions, it is also possible to observe multiarmed spirals, as well as pairs of antispirals with a finite number of arms and pairs [57]. We show examples of this fascinating pattern formation in figure 6. An indepth qualitative analysis reveals that such multiarmed spirals and antispirals are more robust to mobility than singlearmed spirals, persisting not only at small, but also at intermediate mobility rates. In addition to prepared initial conditions, periodic currents (or localized forcing) can also be applied to evoke specific spatial patterns. If a periodic current of all three strategies is applied at the centre of the lattice, target waves can be observed, as reported in [139]. Such phenomena emerge owing to the interplay between local and global system dynamics, in particular, the intermittent synchronization between the periodic current and the global oscillations of the density of the three competing strategies on the spatial grid. In figure 7, we show how the global target wave (top row from left to right) emerges owing to the emergence of resonance between the periodic current and the global oscillations (bottom row from left to right).
The impact of mobility can also be studied in offlattice simulations, where the radius of the interaction range crucially affects coexistence. In particular, a transition from coexistence to extinction is uncovered with a strikingly nonmonotonous behaviour in the coexistence probability [140]. Close to the minimal value of the coexistence probability and under the influence of intermediate mobility, it is even possible to observe transitions between spiral and plane waves over time, as depicted in fig. 2 of [140]. In general, strong mobility can either promote or impair coexistence depending on the interaction range, although diversity is quite robust across large regions of the parameter space [52]. Importantly, these phenomena are absent in any latticebased model, and it is possible to explain them using PDEs similar to equation (3.2). Offlattice simulations of RPS and related evolutionary games are still relatively unexplored. Partly, this is surely owing to significantly higher computational resources that are needed to simulate such systems in comparison with traditional spatial games, yet, on the other hand, the efforts may be well worthwhile as the offlattice setup is closer to some actual conditions than latticebased simulations.
Another interesting phenomenon to consider that is directly related to mobility is epidemic spreading. The effects of epidemic spreading on species coexistence in a spatial RPS game have been studied in [51,54,141]. It has been shown that in the absence of epidemic spreading, there exist extinction basins that are spirally entangled for m > m_{c}, whereas basins of coexistence emerge for m < m_{c}. Moreover, it has been discovered that intraspecies infection promotes coexistence, whereas interspecies spreading fails to evoke the same effect [51,54,141].
To close this section, we note that many more interesting and counterintuitive phenomena are observable when more than three strategies compete for space. In a fourstrategy system, for example, when a twostrategy neutral alliance may emerge, the intensity of mobility is able to directly select the winning solution of the subsystem [142]. More specifically, if the mobility is small the RPStype cyclic dynamics is dominant, whereas if the mobility is large, the twostrategy neutral alliances take over. Further fascinating examples related to cyclical interactions between more than three strategies are reviewed in §5.
3.3. Metapopulation and nonlinear mobility
In §3.2, as in the vast part of the literature, the twodimensional RPS models were implemented on square lattices whose nodes were either empty or occupied by an A, B or C player, and the interactions described by equations (2.1), (2.3), (3.1) involved agents on neighbouring sites. Inspired by the experiments of [2,16,24], an alternative metapopulation modelling approach allowing further analytical progress has recently been put forward [91,136,143]. In the metapopulation formulation, the lattice consists of an array of L × L patches each of which comprises a wellmixed (sub)population of constant size N (playing the role of the carrying capacity) that consists of A, B or C individuals and empty spaces. The RPS game is then implemented according to the general reactions equations (2.1)–(2.4) between individuals within the same patch (intrapatch reactions), whereas the spatial degrees of freedom are granted by allowing the individuals to move between neighbouring patches. At this point, it is interesting to divorce the pairexchange (with rate γ_{e}) from the hopping (with rate γ_{d}) process, according to the scheme (X, Y = A, B, C) 3.3 As shown in [91,144], this leads to nonlinear mobility when γ_{e} ≠ γ_{d} (see also [50,109]). In biology, organisms are, in fact, known not to simply move diffusively, but to sense and respond to their environment. Here, equation (3.3) allows us to discriminate between the movement in crowded regions, where mobility is dominated by pairexchange, and mobility in diluted regions where hopping can be more efficient.
The generic twodimensional metapopulation model of equations (2.1)–(2.4), supplemented by equation (3.3), is ideally suited to capture stochastic effects via size expansion in N [96] and, to lowest order (where all fluctuations are neglected) and in the continuum limit [144], yields the following PDEs 3.4 with γ_{nl} = γ_{d} − γ_{e} and the notation a ≡ a(r, t), b ≡ b(r, t) and c ≡ c(r, t). We note that nonlinear diffusive terms of the form a∇^{2}(b + c) − (b + c)∇^{2}a arise when γ_{nl} ≠ 0. In [91,137,144], the solutions of equation (3.4) have been shown to accurately match the stochastic simulations of the metapopulation model performed using a spatial version of the Gillespie algorithm [97]. It has been shown that the PDEs (3.4) accurately capture the properties of the lattice metapopulation model as soon as the carrying capacity is , whereas their derivation assumes [91,144]. Quite remarkably, it turns out that certain outcomes of stochastic simulations with N = 2−16 are qualitatively reproduced by the solutions of equation (3.4) [145].
In §3.4, we shall discuss how the PDEs (3.4) can be used to derive a CGLE, with an effective linear diffusive term, from which many of the spatiotemporal properties of the spiralling patterns that we have encountered can be predicted. Yet, some intriguing features of the model triggered by nonlinear diffusion cannot be captured by the CGLE. An interesting effect of nonlinear mobility appears in the RPS model of equations (2.1)–(2.4) at low mutation rates when γ_{d} > γ_{e}: as illustrated in figure 8, in this case, the nonlinear mobility promotes the far field breakup of spiral waves and enhances their convective instability [91,144,145].
3.4. Complex Ginzburg–Landau equation
The CGLE is a celebrated and wellstudied nonlinear equation commonly used in the physics community to describe a variety of phenomena spanning from superconductivity, superfluidity, liquid crystals to field theory [146]. After the introduction of a phenomenological timedependent Ginzburg–Landau theory for superconductors [147], the CGLE was derived in the context of fluid dynamics [148,149] and is known for its ability to predict the emergence of complex coherent structures, such as spiral waves in twospatial dimensions, as well as chaotic behaviour. Its fascinating properties have received considerable interest and are the subject of numerous reviews [146,150]. It has recently been realized that many of the spatiotemporal properties of the twodimensional RPS models can be understood by means of a suitable mapping onto a CGLE [3,45,133].
The first attempts to use the CGLE in the context of spatial RPS games have been carried out in [3,45,133] for the model with dominanceremoval reactions (2.1), whereas z = μ = 0, and pair exchange in equation (3.1). At mean field level, this model is characterized by an unstable coexistence fixed point s* with one stable eigenvector, and the dynamics quickly approaches a manifold tangent to the plane normal to the stable eigenvector of s*. In [3,45,133], that manifold was computed to quadratic order around s*. On such manifold, the dynamics becomes twodimensional, and the mean field flows approach the absorbing boundaries of the phase portrait where they linger and form a heteroclinic cycle [87,93]. Exploiting a suitable nonlinear (‘nearidentity’) transformation, effectively mapping (a,b,c) − s* onto the new variables z = (z_{1},z_{2}) and by retaining terms up to cubic order in z, the rate equations (2.5) were then recast in the normal form of a Hopf bifurcation. As explained in [28,133], where details of the derivation can be found, space and movement were finally reinstated by adding a linear diffusive term to rate equations in the z variables yielding a CGLE. It has to be noted that such an approach relies on three uncontrolled approximations:

It consists of approximating heteroclinic cycles by stable limit cycles resulting from a fictitious Hopf bifurcation.

While the new variables z_{1} and z_{2} are assumed to be small quantities, and therefore the mapping is expected to be suitable near s*, its use is not restricted to the vicinity of s*.

Even though mobility is mediated by pair exchange in equation (3.1), which results in linear diffusion in the PDEs (3.2), the nonlinear mapping (a,b,c) − s* → z generates nonlinear diffusive terms that are ignored by the CGLE.
Notwithstanding, this approach has been able to explain many qualitative features of the above model and, upon adjusting one fitting parameter, even obtain approximations for the wavelength of spiralling patterns and the velocity of the propagating fronts [3,45,133]. This treatment was then extended to the case of dominanceremoval and dominancereplacement cyclic competitions with linear mobility and no mutations (p, z, γ > 0, μ = 0) [46,143]. Recently, a class of RPSlike models with more than three species (with dominanceremoval and hopping but no mutations) has also been considered by generalizing this approach [151].
Recently, an alternative analytical treatment has been devised to derive perturbatively the CGLE associated with the generic class of RPS models defined by the reactions equations (2.1)–(2.4), and with mobility mediated by pairexchange and hopping in equation (3.3) [91,144]. This alternative derivation has been carried out within the metapopulation formulation outlined in §3.3. The key observation is that the mean field dynamics of the model defined by equations (2.1)–(2.4) is characterized by a Hopf bifurcation at μ = μ_{H} (see §2) and by stable limit cycles when μ < μ_{H}. A multiscale expansion around the Hopf bifurcation is appropriate to derive a CGLE that aptly describes the fascinating oscillatory patterns arising on twodimensional lattices when μ < μ_{H}. As explained in [91,137], a space and time perturbation expansion in the parameter is performed by introducing the ‘slow variables’ and by expanding the densities in powers of ε [152]. Hence, after a suitable linear transformation (a, b, c) − (a*, b*, c*) → (u, v, w), where w decouples from u and v (to linear order), one writes , and , where the functions are of order . Substituting these expressions into equation (3.4) with , where is a (complex) modulated amplitude, the CGLE is thus derived by imposing the removal of the secular term arising at order , which yields [91,137,144] 3.5 where and, after rescaling by a constant [137], 3.6
It is worth stressing that the CGLE (3.5) is a controlled approximation of the PDEs (3.4) around the Hopf bifurcation point, and it is noteworthy that it involves a linear (real) effective diffusion coefficient. The CGLE (3.5) permits an accurate characterization of the spatiotemporal patterns in the vicinity of the Hopf bifurcation point. For the sake of simplicity, we here restrict σ and ζ into [0,3]. Using the wellknown phase diagram of the twodimensional CGLE [146,153], one thus distinguishes four phases separated by the three critical values (c_{AI}, c_{EI}, c_{BS}) ≈ (1.75, 1.25, 0.845), as illustrated in figure 9a: the absolute instability (AI) phase in which stable spiral waves cannot be sustained (when c > c_{AI}), the Eckhaus instability (EI) phase in which spiral waves are convectively unstable, with spirals' arms that far from the cores first distort and then break up (when c_{EI} < c < c_{AI}), the bound state (BS) phase characterized by stable spiral waves with welldefined wavelength and frequency (when c_{BS} < c < c_{EI}), and a phase in which spiral waves annihilate when they collide (SA phase, when 0 < c < c_{BS}).
The properties of the CGLE (3.5) have also be exploited to gain insights into the properties of the spatiotemporal patterns arising away from the Hopf bifurcation, where the SA phase appears to be generally replaced by an extended BS phase [137,144,145]. Yet, it has also been found that when an Eckhauslike farfield breakup of the spiral waves occurs away from the Hopf bifurcation, very much like in figure 4 (bottom panel; see [144]).
4. Evolutionary games with spontaneously emerging cyclic dominance
Cyclic dominance occurs not only in RPS and related evolutionary games where the closed loop of dominance is explicitly imposed on the competing strategies through the directions of invasions between them (figure 1). Quite remarkably, cyclical interactions may occur spontaneously in any evolutionary game where the competing strategies are three or more, and it is also possible to observe them in twostrategy games if certain playerspecific properties are timedependent [76]. Wellknown examples include the classic public goods game [31], where in addition to cooperators and defectors the third competing strategy is volunteering [66,67,154–157], rewarding [73,74,158–160] or punishing [68,69,161–164]. Further examples include the ultimatum game with discrete strategies [165], the public goods game with pool punishment [71,166–168] as well as the public goods game with correlated positive (reward) and negative (punishment) reciprocity [169].
As in the original RPS game, in these evolutionary games, the spontaneous emergence of cyclic dominance also promotes the coexistence of competing strategies, and through that it stabilizes solutions that would otherwise be unstable. As such, the spontaneous emergence of cyclic dominance is one of the main driving forces behind complex pattern formation, which, in turn, is responsible for the many differences between evolutionary outcomes reported in wellmixed and structured populations. Examples include the stabilization of reward [73] and punishment [68] in structured populations, the evolution of empathy and fairness in human bargaining [165], as well as an oscillatory coexistence of defection and cooperation with the ‘titfortat’ (T) strategy [170–172]. The latter example is particularly instructive, as there is a transition from a threestrategy stationary state to a threestrategy oscillatory state that depends on the cost of the ‘titfortat’ strategy. This phase transition is possible only in a structured population, and because there is a spontaneous emergence of cyclic dominance only between the three competing strategies. The complete phase diagram is shown in figure 10, where the dotted blue line delineates to two threestrategy phases. It is worth emphasizing that an oscillatory state emerges not because of a timevarying interaction network (the importance of which we have emphasized in §31 in the realm of the RPS game), but solely owing to the relations between the competing strategies as defined via the payoff matrix.
Another example that merits attention is the stabilization of punishment in the realm of the public goods game. In general, punishers bear an additional cost if they wish to sanction defectors, and thus face evolutionary pressure not only from defectors, but also from pure cooperators, which effectively become secondorder freeriders [173–175]. Accordingly, in wellmixed populations, punishment is evolutionarily unstable in the absence of additional mechanisms or further strategic complexity. In structured populations, however, punishment alone introduces cyclic dominance such that punishing cooperators are able to invade defectors, defectors are able to invade pure cooperators, and pure cooperators are able to invade punishers [68]. Even more complex evolutionary outcomes are possible in the case of pool punishment [71], where one ‘strategy’ in the closed loop of dominance can be an alliance of two strategies (for further details, we refer to §4.3). Interestingly, the introduction of selforganized punishment, when the fine increases proportional to the activity of defectors, prevents the spontaneous emergence of cyclic dominance and results in a more effective way of punishment [176]. In the latter case, however, the stability of punishers is not directly challenged by pure cooperators (secondorder freeriders), as the punishing activity (and the additional cost of sanctioning) ceases in the absence of defectors.
4.1. Timedependent learning
As we have noted previously, cyclical interactions may occur spontaneously not only if the competing strategies are three or more, but also in twostrategy games if certain playerspecific properties are timedependent. One such example concerns timedependent strategy learning ability of players in the spatial Prisoner's Dilemma game [76]. In a preceding work [177], it has been shown that a limited teaching activity of players after a successful strategy pass modifies the propagation of strategies in a biased way and can promote the evolution of cooperation in social dilemmas. A significantly different effect, however, can be observed if the learning ability of players is timedependent [76]. A temporarily impaired learning activity generates cyclic dominance between defectors and cooperators, which helps to maintain the diversity of strategies via propagating waves, as illustrated in figure 11. Consequently, cooperators can coexist with defectors even under extremely adverse conditions. This result is particularly inspiring because it indicates the possibility of a timedependent playerspecific property to effective replace a third strategy in a closed loop of dominance. Hence, cyclic dominance can emerge only between two competing strategies. Other works on dynamical learning in games with cyclic interactions, where agents sample a finite number of moves of their opponents between each update, have shown how demographic fluctuations can lead to noisesustained cyclic orbits [178,179].
4.2. Voluntary participation
Evolutionary games with voluntary participation were among the first to record the spontaneous emergence of cyclic dominance outside the realm of the traditional RPS formalism. In particular, the coexistence of cooperators and defectors owing to the presence of volunteers (or loners) was initially reported in wellmixed populations [66,67], where the cyclic invasions among the three competing strategies resulted in an oscillatory state. Inspired by Alice's Adventures in Wonderland, where the red queen found that it takes a lot of running to stand still, the emergence of oscillations that were needed to sustain cooperation was dubbed accordingly as the ‘red queen’ effect. Notably, the terms were proposed in the early 1970s [103] as an evolutionary hypothesis based on which organisms must constantly adapt and evolve, not merely to gain a reproductive advantage, but also simply to survive.
Today, volunteering is known as a viable mechanisms to avert the tragedy of the commons [180] within the realm of the public goods game. Hauert et al. [66] studied this phenomenon by assuming that individuals have three available strategies, namely unconditional cooperation, defection and volunteering, which resulted in a RPSlike evolutionary dynamics. In particular, if there are n_{c} cooperators, n_{d} defectors and n_{l} loners in the group, then actually only k = n_{c} + n_{d} individuals are involved in the public goods game. Defectors thus get the payoff rn_{c}/k, where r > 1 is the multiplication factor that takes into account the synergetic effects of a group effort, whereas cooperators gain (rn_{c}/k) − 1 (here −1 is due to their contribution to the common pool, which defectors withhold). At the same time, loners who do not participate in the game, get a fixed small payoff σ, such that 0 < σ < r − 1. Based on numerical simulations and the analysis of the replicator equation, it quickly follows that frequencies of all three strategies exhibit oscillations over time if r > 2. These theoretical results have been confirmed by human experiments performed by Semmann et al. [67], which revealed that volunteering reduces the size of groups engaged in the public goods game, and that cooperation is indeed promoted through timedependent oscillations. The rationale is that defectors outperform cooperators in a large predominantly cooperative group, yet as soon as defectors become the predominant force, it pays more to be a loner. But as loners grow in numbers and the actual size of the group participating in the game decreases, cooperation again becomes viable, and so the loop of dominance closes.
Motivated by the pioneering human experiments of [67], a new experiment has been designed and conducted with a total of 90 undergraduate students from Wenzhou University, who participated in repeated public goods games with volunteering in groups of five. The principal goal was to measure the average frequencies of strategies over time. The payoffs were the same as described in the preceding paragraph, thus designating σ and the multiplication factor r as the two main parameters. The results are summarized in figure 12, which confirm that all three strategies coexist within a wide range of parameter values. However, the timeaveraged frequencies depend on the actual values of σ and r. As shown in figure 12a,b, defectors are most common for σ = 0.5 and r = 3.0, which reverses in favour of loners for σ = 1.5 and r = 3.0, thus experimentally confirming the expected promotion of loners as σ increases. The dominance of loners is even more remarkable for σ = 2.5 and r = 3.0, but shifts towards cooperators for σ = 2.5 and r = 6.5, as shown in figure 12b. This, in turn, confirms the evolutionary advantage of cooperators that is warranted by higher multiplication factors. Interestingly, the outcome of these experiments indicates that the coexistence of the three strategies is more widespread than predicted by theory. For example, all three strategies coexist even for σ > r − 1 (see σ = 2.5 and r = 3.0 case in figure 12b), which may be attributed to the bounded rationality and emotions of participants that are not accounted for in simulations.
Aside from the theoretical explorations of the public goods game with volunteering in wellmixed populations and the abovedescribed human experiments, the public goods game with volunteering in structured populations also received its fair share of attention. In the light of the reviewed results on the spatial RPS game, it is perhaps little surprising that structured populations also maintain the survival of all three competing strategies, but they do so not through the emergence of oscillations, but rather through timewise constant fractions of cooperators, defectors and loners [154]. However, the topology of the interaction network can play a key role, much the same as by the RPS game [118,119]. Specifically, if the interaction network has sufficiently strongly pronounced smallworld properties (there exists a threshold in the fraction of shortcut links that must be exceeded), then the stationary threestrategy phase gives way to an oscillatory threestrategy phase [155,181]. Further strengthening the analogies between the RPS game and the public goods game with volunteering, we note that oscillations in the latter can also be evoked not only through changes in the topology of the interaction network, but also through changes in the payoffs, as illustrated in fig. 1 of [181].
Interestingly, the socalled joker strategy, introduced by Arenas et al. [182], is conceptually different from the loner strategy, but has a similar impact on the evolution of cooperation in social dilemmas. In particular, in finite wellmixed populations oscillations emerge spontaneously, and this is irrespective of the system size and the type of strategy updating [77,182], thus adding yet further support to the relevance of cyclic dominance in evolutionary games.
4.3. When three competing strategies are more than three
If the competing strategies in an evolutionary game are more than two (with some exceptions, as described in §4.1), this opens up the possibility for the spontaneous emergence of cyclic dominance with much the same properties that characterize the classical RPS game. Sometimes, however, three competing strategies are more than three, because some subsystem solutions, typically composed of a subset of the three original strategies, manifest as an additional ‘strategy’. The fourth ‘strategy’ is not exactly equivalent to the original three strategies because it only emerges during the evolutionary process, yet its involvement in the closed loop of dominance is just as important. These rather exotic solutions, which can be observed only in structured populations, have recently been reported in the spatial public goods game with pool punishment [71], and in the spatial ultimatum game with discrete strategies [165].
In the spatial public goods game with pool punishment, we initially have three competing strategies, which are cooperators (C), defectors (D) and pool punishers (P). Quite remarkably, at certain parameter values (see [71] for details), defectors and pool punishers form a stable alliance, which effectively manifests as a strategy, and this strategy forms a closed loop of dominance with cooperators and defectors. Cyclic dominance ensues, where in addition to the two pure strategies C and D, the D + P alliance is the third competing strategy. As a consequence, the characteristic spatial patterns feature propagating fronts made up of all the mentioned competitors, as illustrated in figure 13a.
The peculiarity of the abovedescribed solution indicates that it may not be satisfactory to describe the actual state of a threestrategy system by a single ‘point’ in a ternary phase diagram. Namely just the fractions of the competing strategies do not necessarily determine the actual state of the system in an unambiguous way, and this is because spatiality offers an additional degree of freedom in comparison with solutions that can be observed in wellmixed populations. As described in [71], this new degree of freedom may give rise to an additional ‘strategy’ that is as such not present at the start of the evolutionary process.
Importantly, although such evolutionary solutions may at first glance appear as special and perhaps even limited outcomes of a particular evolutionary game, we emphasize that they may in fact be common place. To corroborate this, we review a completely different evolutionary game, namely the spatial ultimatum game with discrete strategies [165]. Unlike in the public goods game, in the ultimatum game, the focus is not on the aversion of the tragedy of the commons, but rather on the evolution of empathy and fairness [183]. While discrete strategies still cover the whole (p, q) proposal–acceptance parameter plane, the discreetness takes into consideration the fact that in reality these levels are coarsegrained rather than continuous. Thus, instead of the infinite number of continuous strategies, we have N discrete strategies E_{i}, defined according to their (p, q) interval (see [165] for details). If p = q, we have N different classes of empathetic players who can compete with an additional strategy A that is characterized by an arbitrary (p, q) pair. Most crucially, the finite number of competing strategies enables pattern formation, and as illustrated in figure 13b, cyclic dominance may emerge spontaneously where a member in the closed loop of dominance can be an alliance of two strategies. This example, in addition to the previously described one concerning the spatial public goods game with pool punishment [71], thus corroborates the fact that in structured populations three competing strategies can in fact be more than three.
5. Cyclic dominance between more than three strategies
Extensions of the classical RPS game to more than three competing strategies are straightforward (figure 1), and in addition to their obvious appeal from the theoretical point of view [81,94,121,151,184], such games also find actual applicability in describing competition in microbial populations [82] and larger ecological food webs and communities [22,23,185–188]. Based on the review of results concerning the RPS and related threestrategy evolutionary games with cyclic dominance, it is little surprising that the complexity of spatial patterns, and indeed the complexity of evolutionary solutions in general, increases drastically as we increase the number of strategies that form a closed loop of dominance. The spontaneous emergence of defensive alliances and numerous stable spatial distributions that are separated by first and secondorder phase transitions are just some of the phenomena that one is likely to observe.
The reason behind the mushrooming number of stable solutions with the rising number of competing strategies lies in the fact that the solutions of subsystems, where some strategies are missing, can also be solutions of the whole system. The final stationary state can thus be determined not only by the competition of individual strategies, but also by the competition of subsystem solutions (defensive alliances) that are characterized by their own composition and spatiotemporal distribution of strategies (we have already reviewed two such examples in figure 13). Consequently, the general understanding of evolutionary games with many competing strategies requires the systematic analysis and comparison of all possible subsystem solutions, i.e. solutions that are formed by just a subset of all available strategies.
The simplest generalization of the RPS game is towards a fourstrategy cyclic dominance game [47,189–191]. As reported in [47], the formation of spiral patterns takes place only without a conservation law for the total density, and like in the threestrategy case, strong mobility can destroy species coexistence. Cyclic dominance among four strategies also gives rise to socalled neutral strategies, which do not invade each other—an option that is not available among three competing strategies. In the absence of mixing, neutral strategies can easily result in populationwide frozen states, where only those strategies that do not invade each other are present. More precisely, the coexistence of all four strategies is limited to a rather narrow range of invasion rates, whereas, otherwise, the system evolves into a frozen state where only neutral strategies are present [79]. This observation further supports the fact that the topology of the food web alone cannot unambiguously determine the evolutionary outcome of cyclical interactions. In addition, the invasion rates and the interaction range are likewise crucial parameters that determine the transitions from coexistence to uniformity among cyclically competing strategies [188,192,193].
Globally synchronized oscillations in a fourstrategy cyclic dominance game are also possible, as reported in [121] and presented in figure 14. In fact, Rulquin & Arenzon [121] have shown that such coexistence phases require less longrange interactions than threestrategy phases, although intuitively one would expect the opposite. Because the emergence of long distance, global synchronization states is thus possible even when the food web has multiple subloops and several possible local cycles, this leads to the conjecture that global oscillations might be a general characteristic, even for large, realistic food webs.
In addition to the coexistence of all competing strategies, the emergence of ‘frozen’ states, and global oscillations mentioned above, curvaturedriven domain growth is also possible when the evolutionary dynamics is governed by interactions of equal strength among more than three strategies [56,59,60]. In this case, the timedependence of the areas of different domains, which can be quantified effectively by means of the length of domain walls, confirms to the wellknown algebraic law of the domain growth process [194].
5.1. Alliances
The subject that deserves particular attention within the realm of cyclic dominance between more than three strategies is the formation of alliances. As described in the introductory paragraphs of this section, alliances play a key role in the fascinating complexity and multitude of solutions that grace such systems, particularly through the formation of subsystems solutions that together with the pure strategies form the complete solutions of the whole system.
Alliances can emerge in strikingly different ways, although the most common one is the formation of socalled defensive alliances. For example, in a fourstrategy predator–prey system, two neutral strategies can defend each other from the two external invaders [195], hence the name a ‘defensive alliance’. The protective role of an alliance can also be observed when three cyclically dominating strategies are present. In this case, they can protect each other from an external invader by invading the predator of their prey [184]. We also emphasize that cyclically dominant strategies may not necessarily form defensive alliances, even if the governing food web is complex and large. For example, in figure 1d, the strategies RS + KS + SS form a subsystem solution, but this is not an alliance as it is vulnerable to the invasion of an external strategy. On the other hand, the strategies RS + KR + SK form an actual alliance, and a membership in the latter can effectively protect all three strategies within the loop from an external invasion.
Moreover, if the number of competing strategies is high enough, it is also possible to observe the competition between the different alliances [81]. Alliancespecific invasion rates, however, can break the symmetry of the food web, and subsequently only a single alliance survives [196]. The complexity of a sixspecies predator–prey food web with alliancespecific invasions rates also enables noiseguided evolution within cyclical interactions [80], which is deeply rooted in shortrange spatial correlations and is conceptually related to the coherence resonance phenomenon in dynamical systems via the mechanism of threshold duality [197–200].
When it comes to the study of alliances within games of cyclic dominance, the general question to answer is which strategies are capable of forming alliances? As we have reviewed above, two strategies are sufficient to form an alliance, but of course three or even more strategies could also form one. The phase diagram presented in figure 15 demonstrates that, in fact, all the mentioned alliances can be present in the stable solution of the system, depending also on the parameter values that determine invasion and mixing in the studied sixspecies predator–prey system [201]. The key property that determines whether an alliance is viable or not is its ability to invade other alliances or resist such an invasion itself. Similar to the competition among pure strategies, here too the appropriate approach is to measure directly the velocity of invasion between competing domains that contain specific subsystem solutions [63]. The use of special initial conditions, like the one illustrated in figure 16 (remnants of the straight vertical interface that separated two threestrategy cyclic alliances can still be clearly inferred), is thereby very useful and recommended. Unlike pure strategies, however, here we first need to partition the population and wait for the appropriate subsystem solutions to emerge before removing the barriers along the interfaces.
Importantly, the formation of alliances is not always the consequence of specific relations within the governing food web. Alliances may also emerge spontaneously during the evolutionary process, for example owing to payoff relations between the competing strategies that are owing to multipoint (group) interactions. The phenomenon was observed in the spatial public goods game, where in addition to cooperators and defectors both pool (O) and peer (E) punishers competed for space [202]. Figure 16 illustrates an example of the spontaneous emergence of two threestrategy cyclic alliance. Notably, either the D + C + O or the E + O + D alliance can be stable solutions of the whole system at specific values of the punishment cost and fine. But if they meet, as displayed in figure 16, the E + O + D alliance will invade the D + C + O alliance. Remarkably, the invasion process between the two alliances is effectively the same as it would be between two pure strategies.
As by the competition among pure strategies [3], by the competition of alliances too the role of mobility (or diffusion) is likewise important. Within the framework of a ninespecies predator–prey model [184], it has been shown that below a critical mobility rate, the system exhibits expanding domains of three different defensive alliances, each consisting of three cyclically dominant species. Owing to the neutral relationship between these three defensive alliances, a votermodeltype coarsening starts [203], and after a sufficiently long time, only one alliance survives if the system has finite size. Above the critical mobility rate, however, the three defensive alliances can all coexist [44]. In a way, the effect of high mobility is thus the opposite to the one observed in the RPS game. There high mobility rates jeopardize diversity and lead to the survival of a single strategy [3], whereas the coexistence of defensive alliances may receive unexpected support. Further related to the impact of mobility on the survival of alliances, a recent study has demonstrated that mobilitydependent selection can prefer either a threestrategy cyclic alliance or a neutral pair of twostrategy alliances within a fourstrategy system [142].
At this point, we again emphasize that it is not enough to simply separate random mixtures of selected strategies that are involved in particular alliances and monitor the evolution. Before the barriers along the interfaces are removed, a sufficiently long relaxation time has to be taken into account, which allows the actual subsystem solutions (rather than just the random mixtures of the involved strategies) to emerge. Only then will the actual competition between the alliances unfold and give correct insights into the power relations between them. In the absence of such a procedure, it may easily happen that one of the considered subsystem solutions cannot evolve fast enough, for example owing to requiring a longer relaxation time than another subsystem solution, and thus it becomes impossible to properly compare their viability. A specific example to that effect concerns a fivestrategy predator–prey system [63,204], where if the mobility is strong enough the dominance between the competing alliances can vanish simultaneously. This phenomenon is then also accompanied by a divergence in the density fluctuations of individual strategies. In the absence of mobility, however, the dominance between the alliances does not vanish simultaneously, and only a sharp peak in the density fluctuations can be observed instead of the powerlaw divergence. The phenomena observed in the realm of this fivestrategy predator–prey system in fact outline a more general type of behaviour, namely the mobilityinduced reversal of the direction of invasion between the competing alliances, thus highlighting yet again that the topology of the food web does not always (in fact, it does so only rarely) determine the final evolutionary outcome in structured populations.
6. Conclusion and outlook
As we hope this review clearly shows, cyclical interactions are a fascinating subject, which have the power to captivate with the complexity and beauty of the governing evolutionary dynamics, and which in addition to this has many reallife applications that we have only begun to fully appreciate. We have reviewed main results in wellmixed populations with the focus on oscillatory dynamics and the extinction to absorbing states owing to finite system size. We have also reviewed results on the classical RPS game in structured populations. We have first focused on the important role of the topology of the interaction networks, which may induce oscillatory states, although the latter can also be evoked by changes in the game parametrization. We have also focused extensively on mobility, which has many practical implications as in reallife prey and predators are frequently on the move in order to maximize their chances of survival. We have emphasized that mobility can either promote or jeopardize biodiversity, and it may give rise to fascinatingly rich pattern formation, including spiral and target waves, multiarmed spirals with two, three and four arms, as well as antispirals. Mobility can also affect the basins of coexistence and extinction through intraspecies and interspecies epidemic spreading. An alternative metapopulation modelling approach is also reviewed, where the distinction of pairexchange and hopping processes allows us to discriminate movement in crowded and in diluted regions. The resulting nonlinear mobility promotes the far field breakup of spiral waves and enhances their convective instability. Within this metapopulation formulation, we have also derived perturbatively the CGLE.
In addition to the review of the wellmixed and spatial RPS game, we have also given special attention to evolutionary games where cyclic dominance emerges spontaneously, for example between only two competing strategies through timedependent learning, among three strategies in public goods game with volunteering, punishment or reward as well as by means of the introduction of the joker strategy. We have reviewed how alliances may effectively act as strategies and close the loop of dominance in threestrategy evolutionary games, including the public goods game with pool punishment and the ultimatum game with discrete strategies. From there onwards, we have surveyed cyclic dominance when the competing strategies are more than three, where we have emphasized the possibility of competition between alliances. We have highlighted the importance of the correct preparation of special initial conditions that allow proper monitoring of the invasion relations between the alliances, and we have given prominent exposure to the fact that alliances may also form spontaneously, without support from specific relations within the governing food web.
There are still unexplored problems related to cyclic dominance in evolutionary games that merit further attention. While physicsinspired studies account for the majority of recent theoretical advances on this topic, there also exist much experimental and theoretical work that was outside the scope of this review. Competitive intransitivity and the coexistence of species in intransitive networks [205,206], as well as communityscale biodiversity in ecological systems with a large number of species [207] and the governing large competitive networks [185–187], are just some of the topics that we did not cover in detail. While we have mentioned several experiments and empirical data that corroborate the importance of cyclical interaction in the Introduction, covering this in detail could easily amount to an independent review. In addition to the oftencited examples mentioned in the Introduction, most recent experimental research suggests that we may have only just begun to scratch the surface of the actual importance of cyclic dominance in nature [83,84]. It would be desirable for the fundamental theoretical advances to become more in sync with the experiments by means of more dedicated and precise mathematical modelling. It is worth noting that cyclic dominance could be crucial not just to understand biodiversity, but also social diversity [208–210].
In terms of future fundamental research, we would like to point towards the basically unexplored subject of cyclic dominance in group interactions. In social dilemma games, the transition from pairwise to group interactions [211,212] brings with it many qualitative changes in the evolutionary dynamics (see [31] for a review), and the reasonable expectation is that the same would also hold true in games of cyclic dominance. We have recently considered the impact of different interaction ranges in the spatial RPS game, and found that the transition from pairwise to group interactions can decelerate and even revert the direction of the invasion between the competing strategies [193]. These results thus indicate that, in addition to the invasion rates and other properties of cyclic dominance games already reviewed here, the interaction range is at least as important for the maintenance of biodiversity among cyclically competing strategies, and is thus surely deserving of further attention.
Motivation for future research can also be gathered from coevolutionary games [29], where cyclical interactions can still be considered as being at an early stage of development. While initially many studies that were performed within the realm of social dilemma games appeared to be valid also for games that are governed by cyclical interactions, recent research has made it clear that this is by far not the case. The incentives are thus clearly there to reexamine the key findings, which were so far reported only for pairwise games on coevolutionary networks, also for games that are governed by cyclical interactions. We conclude by noting that devising and studying cyclic dominance systems with greater practical applicability and predictive power, including more complex interaction networks and larger numbers of competing strategies, should also be well within the scope of viable forthcoming research.
Funding statement
This work was supported by the Hungarian National Research Fund (grant K101490), the National Natural Science Foundation of China (grants no. 61203145 and 11047012) and the Slovenian Research Agency (grants no. J14055 and P50027). B.S. is thankful for the support of an EPSRC studentship (grant no. EP/P505593/1).
Acknowledgements
The authors gratefully acknowledge the fruitful contributions of many collaborators to part of the research reviewed here.
 Received July 7, 2014.
 Accepted August 22, 2014.
 © 2014 The Author(s) Published by the Royal Society. All rights reserved.