## Abstract

Evolved natural systems are known to display some sort of distributed robustness against the loss of individual components. Such type of robustness is not just the result of redundancy. Instead, it seems to be based on degeneracy, i.e. the ability of elements that are structurally different to perform the same function or yield the same output. Here, we explore the problem of how relevant is degeneracy in a class of evolved digital systems formed by NAND gates, and what types of network structures underlie the resilience of evolved designs to the removal or loss of a given unit. It is shown that our fault tolerant circuits are obtained only if robustness arises in a distributed manner. No such reliable systems were reached just by means of redundancy, thus suggesting that reliable designs are necessarily tied to degeneracy.

## 1. Introduction

One remarkable feature of many biological systems is the presence of a high degree of robustness against perturbations. Such robustness appears at multiple scales (Alon *et al*. 1999; ,Tononi *et al.* 1999; ,Edelman & Gally 2001; ,Gibson 2002; ,Krakauer & Plotkin 2002; ,Li *et al*. 2004; ,Jen 2005; ,Wagner 2005). Specifically, it is often found that temporal failure or permanent loss of some components has very often little or no impact on overall performance. In this context, such entities as a whole are able to cope with a changing world even under the loss of single units. A standard illustration of such robustness (or fault tolerance) is provided by gene knockouts through directed homologous recombination. In a large number of cases (close to 30%), little or no phenotypic effects are observed (,Melton 1994). What is more surprising, it was shown that the mechanisms underlying such reliable behaviour are not based on redundancy. By redundancy, we refer to the presence of multiple copies of a given component: the failure of one of them would be compensated by another identical (isomorphic) copy. Instead, robustness in biology is largely associated with a distributed property, which has been dubbed either *degeneracy* (Tononi *et al*. 1999; ,Edelman & Gally 2001) or *distributed robustness* (Wagner 2005). By degeneracy, we refer to ‘the ability of elements that are structurally different to perform the same function’ (,Tononi *et al*. 1999; ,Edelman & Gally 2001).

The problem of robustness was a hot topic in the 1950s, in parallel with the design of the first electronic computers. Back then, vacuum tube technology was rather unreliable and John von Neumann and others (von Neumann 1952; ,Cowan & Vinograd 1963) explored the problem of how to design reliable computers from unreliable elements. Most of this work was based on digital designs described in terms of logic gates (,figure 1*a*). The main conclusion from these studies was that a high degree of redundancy was required in order to achieve such a goal. Since nature seems to deal with faulty behaviour by using non-redundant mechanisms, something different must be at stake. Exploring the problem of degeneracy involves a number of difficulties. Since redundancy does not explain robustness (at least not most of it), we cannot understand the problem in terms of repeated, dissociated pieces. In this paper, we want to address this problem using evolved synthetic circuits performing simple computations.

Digital and switching circuits have been widely used in modelling gene and signalling networks (Kauffman 1962, ,1993; ,Wuensche & Lesser 1992; ,Mendoza & Alvarez-Buylla 1998; ,Mendoza *et al*. 1999; ,Astor & Adami 2000; ,Kauffman *et al*. 2003; ,Solé *et al*. 2003; ,Weiss *et al*. 2003; ,Sauro & Khodolenko 2004; ,Simpson *et al*. 2004; ,Alvarez-Buylla *et al*. 2006; ,Braunewell & Bornholdt 2007; ,Willadsen & Wiles 2007), as well as other more general problems concerning the evolution of technology (,Arthur & Polak 2006). Evolved circuits provide a good framework where our questions can be explored in a sensible way (,Koza 1992; ,Miller *et al*. 2000). These systems, resulting from artificial evolution, allow one to obtain new designs without direct human intervention, in many cases displaying a higher efficiency (,Koza 1992). Our goal here is the generation of digital circuits by evolution under different external conditions, with the purpose of exploring the emergence of fault tolerance and how it relates to redundancy and degeneracy. Although the digital metaphor has some limitations, it has been widely used in computational systems biology to address very diverse types of questions. It seems to define an appropriate level of description to the switching behaviour found in many biological systems, from gene regulation to cell signalling (,figure 1*b*,*c*). The study of the behaviour of these models has been shown to provide deep insight into the origins and importance of robustness (Bornholdt & Sneppen 2000; ,Klemm & Bornholdt 2005; ,Ciliberti *et al*. 2007; ,Fernandez & Solé 2007). In this context, although previous studies have shown that robust structures emerge as a consequence of evolutionary rules, the exact origin of such robustness is often missing. Here, using the formal definitions introduced by ,Tononi *et al*. (1999) and ,Edelman & Gally (2001), we show that robust designs are achieved by means of distributed robustness.

## 2. Measuring distributed robustness

A first step before we present our results on evolved networks is to properly define a set of quantitative measures of network redundancy and degeneracy. To this goal, we will make use of previous measures from Tononi *et al*. (1999) and ,Edelman & Gally (2001). These information-based measures can be quantified using statistical measures of entropy and mutual information (,Ash 1965; ,Adami 1998). We build these measures on a network *X* of *Z* interacting units where some computation is being performed. This is the case of logic circuits, such as the one shown in figure 2*a*. Here, each unit is a NAND gate. The reason for choosing this particular gate is that it allows one to build *any* possible digital circuit. Specifically, together with the so-called NOR gate, it is one of the two sole sufficient operators that can be used to express all of the Boolean functions of propositional logic (http://en.wikipedia.org/wiki/Shefferstroke).

Several measures can be defined on *X* based on information theory. An appropriate combination of them allows one to properly define robustness and measure it. The basic measure of information theory is the entropy, defined as (Ash 1965; ,Adami 1998)(2.1)for one single variable *x*, and similarly we have(2.2)for two variables *x* and *y*. Here, *p*(*x*) is the probability distribution for the possible values of *x* and *p*(*x*, *y*) is the joint probability distribution associated with (*x*, *y*)-pairs. From these basic measures, it is possible to define the mutual information *I*(*X*, *Y*) in terms of the entropies as(2.3)Mutual information is the key quantity in theoretical approaches to the study of communication systems (Ash 1965; ,Adami 1998). It measures the information transmitted through the underlying channel. It has also been used as a measure of complexity, since it also describes the presence of (non-trivial) correlations (,Solé & Goodwin 2001).

In order to define robustness, we need to provide a quantitative measure where interdependencies among not only elements but also subsets are properly weighted. Such measure must be able to quantify to what extent information transfer among different parts overlaps. Since the success of a given computation is tied to the generation of a right output, the definition of degeneracy (distributed robustness) must somehow incorporate the relation between different subsets and the set of output units (Tononi *et al*. 1998).

In their analysis of robustness, Tononi *et al*. (1999) define the degeneracy *D*_{Z} of the system as(2.4)which we adopt here as our definition of robustness (and in the following we will use both terms as equivalent). Here, represents the *i*th subset of *k* elements, which is possible to build from the *Z* elements of the network and *O* is the subset of elements which form the output layer. The average 〈 〉 is computed over all the possible subgroups with size *k* in which we can divide the system. As defined, it compares the average information transfer between every subset of size *k* and the expected value of the whole information between system and output, weighted by *k*/*Z*. A different (and more convenient) way of writing this function reads(2.5)In this expression, the balance of mutual information takes into account the possible overlapping of the information being processed by a subset formed by *k* elements and the rest of the network, i.e. . In the case of two independent subgroups, the balance is simply . This case corresponds to the lower bound of degeneracy. In any other case, it measures the overlap between subgroups. The upper bound corresponds to the extreme case of full degeneracy. In this case, the mutual information between one subgroup of the network and the output layer must be similar to the mutual information between the rest of the network and the output layer and similar to the mutual information between the total network and the output layer:(2.6)Building a network with such extreme degeneracy is likely to be impossible, but considering this situation it is possible to define an upper bound for degeneracy as(2.7)which is obtained from (2.4) using the condition given in ,(2.5).

These measures allow one to unambiguously determine the presence and impact of degeneracy, but they are also involved and computationally costly. For the example chosen in figure 2, the mutual information for the subset (gates 1 and 2) is 〈*I*〉=0.21. This value means that part of the computation performed by the network from the input to the output involves some overlap between the subset and the rest of the network (gate 3). In other words, two subsets of the network that are structurally different perform, at least partially, the same computation process. To compute the degeneracy values with (2.4), we need to calculate the average value of the mutual information for all the possible subsets for a given size (in the example if the size is 2, there are three possible subsets depicted in ,figure 1*b–d*) and perform the sum over all possible sizes, from *k*=1 to *Z*. Such a scenario implies that, even for small networks, we require a very large number of subset combinations to be considered. This limits the total size of circuits that can be used in our analysis, which in our study is limited to *Z*=15.

Additionally, we can also estimate the redundancy of our system using the following definition (Tononi *et al*. 1999):(2.8)This expression measures the overlap of the information processed by one given element and the rest of the network. If the different elements of the network are independent, then . Otherwise, some of the elements of the network are redundant.

Finally, a complexity measure can be also defined as (Tononi *et al*. 1999)(2.9)This expression measures the level of coherent integration of the different parts of the system (Tononi *et al*. 1998). It takes into account the average mutual information between each possible subset and the rest of the network. If the different subsets have lower values of mutual information , this implies that there is lower overlap between the information processed by each subset and the rest of the network. In this case, the network has a low integration and does not work as a whole, but as a set of more or less independent parts. This case corresponds to small values of *C*_{Z}.

To illustrate the estimation of degeneracy as defined above, we can use a simple logic network as the one shown in figure 2*a*. It is formed by five NAND gates.

Table 1 shows all possible states for each gate for each input combination. Gates G_{4} and G_{5} define the output layer. To compute the degeneracy values, we must build all the subsets of NAND gates. It is possible to define three subsets of one gate each, (*i*=1, 2, 3), three subsets of two gates (*i*=1, 2, 3) as displayed in figures 2*b–d* and one subset of three gates . Here, we illustrate the calculation method for one of these subsets of size two, i.e. . First, we must calculate the different entropy values using the standard definitions (2.1) and ,(2.2). ,Table 2 shows the different states for the subset , shown in figure 2*b*.

In table 3, we give the probabilities associated with the different possible input combinations and the associated entropy calculated directly from the possible states given in ,table 2.

Finally, different mutual information values can be computed from the previous measures. The results are summarized in table 4.

## 3. Evolving circuits

Different evolutionary rules have been used for the synthesis of circuits (Iba *et al*. 1997; ,Yu & Miller 2001; ,Arthur & Polak 2006; ,Banzhaf & Leier 2006). Some of these methods have been able to produce systems with high levels of fault tolerance and robustness. In our study, the circuits start from a randomly wired set of NAND gates. These circuits evolve until they are able to implement a certain *N*-input, *M*-output target binary function *ϕ*, this function being a member of the set of possible Boolean functions described as a mapping(3.1)with . In our study, we will use *N*=3 and *M*=2. The only topological limitation considered here is that there are no backward connections, in order to avoid temporal dependencies. In other words, only downstream effects are considered. The evolutionary rules follow previous work on circuit evolution (Miller & Hartmann 2001) with additional selection constraints in order to canalise the evolution process in two different scenarios: evolutions using correct computation and a fault tolerance measure as an additional constraint.

### 3.1 Selection for accurate computation

The goal of this selection using as fitness measure the correctness of the Boolean computation being implemented. It is thus defined in terms of the matching between the desired target function with no further constraints. The idea here is to see whether circuits that just correctly perform the desired computation are also robust *for free*. The target functions *ϕ* are chosen at random: the set of outputs for each input combination are generated using 0 and 1 with equal probability. The steps of the algorithm are the following.

Create a random generated population formed by

*S*individuals (candidate circuits)*X*_{1}, …,*X*_{S}. Each one of these individuals is formed by*Z*randomly wired gates, with no backward connections. All circuits start with*Z*=5 NAND gates. In each one of these nodes, there is a NAND gate of two inputs and one output. This population constitutes the starting generation. Here, given the computational constraints, we fixed the number of candidate solutions to*S*=10.The behaviour of each individual solution is simulated for the 2

^{N}possible inputs, comparing its outputs with the target function outputs. The fitness for the*k*th circuit*X*_{k}is defined as(3.2)where is the set of outputs of the circuit for the different inputs;*M*is the number of output bits; and is the set of (expected) target function outputs.The individual with greater fitness is chosen to create a new generation formed by the selected individual and

*S*−1 random mutations of it. The random mutations (always respecting the backward patterning) can be: (a) elimination of an existing connection, with probability*E*_{c}, (b) creation of a new connection with probability*C*_{c}, (c) elimination of a node (gate removal) with probability*I*_{n}, and (d) creation of a new node (gate addition) with probability*C*_{n}. Here, we use*E*_{c}=0.8,*C*_{c}=0.8,*I*_{n}=0.3 and*C*_{n}=0.6.Repeat step (ii) until a fitness

*F*=1 is reached.

### 3.2 Selection using fault tolerance

This algorithm differs from the previous one in that selection of the most optimal individual of each generation introduces fault tolerance as an additional requirement. Fault tolerance *ρ*_{k} is measured as(3.3)where is the output set of the circuit for the different inputs and is the output set of the *k*th circuit under a perturbation of the *p*th node. For each input, all the gates are in a binary state 0 or 1. The perturbation consists of the inversion of the logical level of the gate located at the *p*th node. This perturbation is applied for each node *p*.

In this case, the individual chosen to create a new generation must satisfy two conditions, namely greater fitness and higher fault tolerance. This additional condition of higher fault tolerance on the evolutionary process is prevailing (i.e. we give priority to *F*_{k} on top of *ρ*_{k} when *F*_{k}<1). The selection process, in this case, does not stop when *F*=1. Once *F*=1, the individuals can continue evolving until *ρ* reaches a stable value. In this approach, we want to compare the final designs with those resulting from the first evolution approach. Looking at the final circuits generated, we can measure redundancy and degeneracy and see how they contribute to the network robustness.1

## 4. Results

In figure 3, we show two examples of the synthetic circuits obtained from the two previous algorithms. ,Figure 3*a* shows a circuit obtained from fault tolerance selection, with a final value *ρ*=0.944. For the same Boolean function, figure 3*b* shows the corresponding outcome of the first evolution strategy with a much lower fault tolerance of *ρ*=0.54. As a general trend, we have found that evolved circuits from selection using just correct computation are typically smaller than the corresponding circuits selected using a fault tolerance evolutionary dynamics. This is an expected result, since it seems clear that robustness against gate failure must require some kind of internal capacity of reorganization. In order to see what emerges in terms of robustness, we measured redundancy and degeneracy in a set of evolved circuits under both types of selection pressures.

In figure 4*a*, we show the relationship between redundancy and degeneracy (our measure of robustness) for our evolved circuits under both modes of evolution. The figure clearly shows that circuits evolved under properly computing the Boolean function *ϕ* have diverse levels of redundancy but very small degeneracy. Two relevant implications of this result can be obtained. The first is that an active selection for robustness may be required in order to achieve *highly* reliable designs. The second is that circuits obtained under selection for fault tolerance can achieve large levels of robustness and that such robustness is distributed, but requires some amount of internal redundancy (as shown by the rapid increase of *D*_{Z} at high *R*_{Z} values). Let us note that the circuit with greater fault tolerance has a degeneracy value of *D*=6.28, closer to the upper bound value of obtained from (2.6).

These results support the view that biological designs, which are expected to experience different sources of noise and perturbation, make use of degeneracy, instead of redundancy, in order to properly function. As far as we know, it is actually the first accurate demonstration of such relationship based on a well-defined measure of robustness. Moreover, as shown in figure 4*b*, there is a growing trend relating fault tolerance and the system size achieved through the evolutionary dynamics. Specifically, a power law increase, i.e. *ρ*∼*Z*^{γ} with *γ*≈4. This trend is highly nonlinear, with a rapid increase in reliability as *Z* grows for conditional selection. This implies that high increases in robustness can be achieved by properly adding a single new element to the system.

In figure 5*a*,*b*, we also compare our measures of robustness and fault tolerance with the internal organization of the circuits as measured in terms of complexity. In our circuits, we see that large levels of complexity and circuit integration closely follow high degeneracy levels. This is consistent with previous findings using neural networks (Tononi *et al*. 1998). Such high complexity levels indicate that circuits evolve towards structures in which the different parts act with greater levels of coherent integration.

As figure 5*a* shows, an increase in degeneracy implies an increase in the complexity of the circuits, although a saturation is observed beyond some point (probably due to the small circuit sizes, which does not allow further increases). Such a relation does not happen with the redundancy (not shown): high redundancy levels are not consistent with more complex circuits. Similarly, complexity and fault tolerance are related, although the tendency towards higher fault tolerance with network integration is more obvious in circuits obtained through fault tolerant evolution (figure 5*b*).

## 5. Discussion

The evolution of complex life forms seems to be inextricably tied to robustness. The problem of how complexity, modularity, adaptation and reliability are connected is an old one (Conrad 1983) but far from being closed (,Hogeweg 2002; ,Lenski *et al*. 2003; ,Wagner *et al*. 2007). Capturing the details of such relations is a difficult task, and theoretical approaches require strong simplifications that are typically based on a computational picture of the system under consideration. In this paper, we have explored the possible origins of robustness in networks performing computations by using evolved artificial circuits. Although the approach taken is not a realistic biological implementation, it captures some of the logic of cellular networks at least at the level of computation. Nevertheless, our main goal was to determine the possible forms of reaching reliable systems under different selection pressures and understanding how robust designs can be obtained. Given the two potential origins of robust responses, namely either redundancy or degeneracy, we measured the resulting structures in order to determine what are the contributions of each to the observed levels of fault tolerance.

As found in biological systems, we can see that the origins of robustness against the failure of a given element are largely associated with a distributed mechanism of network organization. Both degeneracy and complexity (a measure of network integration and coherence) have been shown to reach high levels provided that redundancy is high enough. In this context, the use of an explicit measure of network reliability based on information exchanges between different subparts allows us to reach well-defined conclusions. Redundancy, on the other hand, is shown to be less relevant to the fault tolerant behaviour of our circuits. However, high redundancy might be needed in order to properly build degeneracy into the system, although what is likely to happen here is that an inevitable, positive correlation is at work.

An important point to be made here concerns the levels of fault tolerance achievable under the first selection criterion. Although maximal levels of fault tolerance have been found in connection to large degeneracy, high levels can also be achieved without explicitely introducing *ρ* in the selection process. This is shown for example in figure 5*b*, where we can see (triangles) that some networks achieving the correct computations are also fault tolerant. Although maximal levels might be interesting for designed systems, the implications for evolved networks are clear: fault tolerance might emerge ‘for free’ as a consequence of the evolutionary rules. This seems consistent with our previous work on evolving networks suggesting that the generative rules of network complexity might pervade many of their desirable functional traits (Solé & Valverde 2006, ,2007).

## Acknowledgments

We thank Alfred Borden and the members of the Complex Systems Lab for useful discussions. This work has been supported by the European Union within the 6th Framework Program under contracts FP6-001907 (Dynamically Evolving Large-scale Information Systems), FP6-002035 (Programmable Artificial Cell Evolution), by the James S. McDonnell Foundation and by the Santa Fe Institute.

## Footnotes

↵In our numerical experiments, the specific set of parameters used was chosen in such a way that evolution was fast enough to be efficient. Other combinations provided similar results but displayed slower convergence.

- Received June 3, 2008.
- Accepted July 11, 2008.

- © 2008 The Royal Society