1 Introduction
The performance and efficiency of large distributed systems, such as open peertopeer networks which any user can join, often critically depend on cooperation and can suffer if users behave selfishly, e.g.: consume but not contribute resources [29], choose routes [40] and neighbors [9] to optimize their personal benefits, etc. Noncooperative behavior can also affect security. For example, if security mechanisms such as the installation of antivirus software are employed just for selfprotection, a virus may spread further than in cooperative environments [10], while at the same time increasing global security investment costs [2].
The Price of Anarchy (PoA) [27] is a gametheoretic concept which allows to assess to which extent a distributed system is affected negatively by noncooperative behavior. Essentially, the PoA compares the optimal social welfare resp. cost to the welfare resp. cost in the worst Nash equilibrium: an equilibrium in which no selfish agent, henceforth called player, has an incentive to change its behavior. If a distributed system has a large PoA, this means that the performance resp. cost can be far from optimal: the system may require a redesign or at least strong incentive mechanisms.
However, Nash equilibria are often not a good model for realworld distributed systems, for several reasons. In particular:

Dynamic character: Distributed systems typically are not based on “oneshot games” but rely on dynamic interactions over time: e.g., peers (i.e., players) in peertopeer systems such as BitTorrent interact repeatedly, for example using titfortat strategies, leading to repeated games [49].

Local information: Players in distributed systems often only have local information about the network, based on interactions (e.g., with neighboring players). Obtaining global information is typically infeasible, especially in largescale distributed systems.

Bounded resources: Players typically also have only limited resources, e.g., in terms of memory or in terms of the complexity of the kind of algorithms they can execute.
This paper is motivated by the desire to extend the concept of price of anarchy to account for these important characteristics of distributed systems. In particular, we aim to port the PoA to evolutionary games and local information scenarios: games with simple, memoryless players which interact repeatedly and locally, and can update their strategies over time. This is timely, and the research community is currently discussing alternatives to Nash equilibria such as BayesNash equilibria [41] for games with incomplete information; at the same time, it is believed that such extensions are complex due to having to take into account players’ belief systems, amongst other issues, and thus introduce major research challenges [39].
In fact, analyzing equilibrium selection in stochastic processes as described by evolutionary games is already challenging in 2player games in a population with pure strategies [13, 12]. Games on graphs, while also highly useful in verification and synthesis of (possibly distributed [38]) reactive systems [6, 32], are often particularly difficult, also when considering evolutionary games [1], due to the additional dependencies on the possible interactions.
1.1 Our contributions
This paper extends the notion of price of anarchy to evolutionary games, introducing the evolutionary Price of Anarchy (ePoA). In particular, we are interested in the equilibrium behavior of simple memoryless players, which repeatedly and locally interact on a graph. The ePoA is essentially a framework and can be used to study different evolutionary dynamics (the parameters to our framework) and ways in which players evolve their strategies.
To shed light on how the evolutionary perspective can affect the conclusion on the possible impact of selfish behavior in distributed systems, we consider a concrete case study: the wellknown virus propagation game introduced by Aspnes et al. [2]. We present an analysis of the evolutionary dynamics of this game for the three fundamental dynamic models (related to genetic evolution and imitation dynamics) and different basic network topologies (cliques, stars, and beyond). Interestingly, while the analysis of such evolutionary games is challenging in general, we are able to provide an exact characterization of the longrun frequencies of configurations for these scenarios.
We make several interesting observations. We find that the evolutionary dynamics of this game give rise to a rich variety of behaviors. In particular, the ePoA can be significantly worse than the classic PoA, for reasonable (i.e. not too high) mutation rates. We also find that Nash equilibria are not always the most frequent, i.e., abundant, states, and different processes can feature significant offequilibrium behavior.
Our analytical results are complemented with simulations, also of more complicated topologies.
1.2 Organization
The remainder of this paper is organized as follows. Section 2 introduces preliminaries. We present the concept of evolutionary price of anarchy in Section 3 and study its application in the virus inoculation game in Section 4. Section 5 reports on simulation results on more complex topologies. After reviewing related work in Section 6, we conclude our work in Section 7. To improve readability, some technical details only appear in the Appendix. More detailed numerical results and our implementation are available upon request.
2 Preliminaries
Before launching into a full description and analysis of our model, we first revisit the virus inoculation game which will serve us as a case study in this paper. We will also give a short introduction to evolutionary dynamics, evolutionary games and evolutionary graph theory.
2.1 The virus inoculation game
In the classic oneshot virus inoculation game [2], nodes must choose between installing antivirus software (inoculating themselves) at a cost, or risk infection by a virus that spreads from a random location and can reach a node if there is a path of not inoculated nodes in between. The network is modeled by an undirected graph with nodes. Nodes correspond to players in the game. Every node is equipped with a strategy that denotes its propensity to inoculate itself. If is 0 or 1, the strategy is called pure. Every node takes an action according to its strategy; the overall configuration of nodes is reflected in the strategy profile . Inoculation costs . After everyone has made their choice, the adversary picks a random node as the starting point of infection. The virus propagates through the network, infecting all noninoculated nodes that have a direct path to an infected node. That is, propagation happens on an ”attack graph” , where inoculated nodes have been removed, and only insecure nodes remain. Being infected then comes with a cost . Inoculation prevents infection as well as further virus transmission by the inoculated node.
The cost of a mixed strategy for a node in this model is therefore
(1) 
where
is the probability of node
being infected given the strategy profile and the condition that did not inoculate. The goal of each player is to minimize its own cost, while it does not take the resulting social cost to the system in total into account. This social cost is simply(2) 
Aspnes et al. then showed the following characterization of pure equilibria (for the proof and the extension to mixed equilibria, cf. [2]):
Corollary 1 (Characterization of pure equilibria).
Fix and , and let the threshold be . A strategy profile is a pure Nash equilibrium if and only if:

Every component in the attack graph has at most size .

Inserting any secure node and its edges into yields a component of size at least .
2.2 Evolutionary dynamics and games
Game theory considers individuals that consciously aim to reach the best outcome for them in a strategic decision situation. Its classic framework, using concepts like Nash equilibria usually makes some key assumptions about the players’ rationality, their beliefs and cognitive abilities. In contrast, evolutionary game theory, as a generic approach to evolutionary dynamics [42, 43, 20], considers a population of players with bounded rationality instead. Each player adopts a strategy to interact with other population members in a game. The players’ payoffs from these interactions – which depend on the actions of the coplayers and therefore on the abundance of different strategies – are considered to be their evolutionary fitness. Success in the game is translated into reproductive success: good strategies reproduce faster, whereas disadvantageous strategies go extinct. In a nutshell, evolutionary game theory describes dynamics that are dependent on the frequency of different strategies in a population.
The evolutionary dynamics depends on the setup and structure of the population, the underlying game, and the way strategies spread. There is a distinction to be made between deterministic dynamics in the limit of infinite populations [19], described by differential equations ([20]), and stochastic dynamics in the case of finite populations [22] that take into account the intrinsic randomness of evolutionary processes. Under appropriate conditions, stochastic models however approach their deterministic counterparts as the population size becomes large [45].
Furthermore, the way strategies spread crucially depends on the way members of the population are connected to each other. Evolutionary graph theory considers different population structures and how the dynamics are changed by changing the underlying graph [28]
. Vertices now represent the members of the population, and edges the spatial/social connections between them. This is a natural model for multiplayer systems, as interactions then happen only between neighbors. The stochastic process is described by a Markov chain on the states of the graph. However, analytical results on these properties in games on arbitrary graphs are difficult to obtain in general; many of the fundamental underlying problems are computationally hard (e.g.,
[21]).3 The Evolutionary Price of Anarchy
It may not be realistic to assume that nodes in a game such as the virus game in [2] will have perfect information in the first place. In large networks, it is unlikely that nodes separated by many neighbors would know and each others’ decision and would then optimally react to this large amount of information. Rather, it is more natural to think that nodes only have local information. They can see their neighbors’ most recent choices and react to them only when updating their strategy, while being unaware of the choices of nodes with a higher degree of separation. To model this, we first introduce an evolutionary virus inoculation game and three different kinds of stochastic evolutionary dynamics. We then define a general notion of the evolutionary price of anarchy.
3.1 The evolutionary virus inoculation game
We consider an evolutionary process on a static graph with the set of vertices (players) and the set of edges. The vertices are occupied by players in the game, and we say that two nodes/players are neighbors if there is an edge connecting them. One iteration of the evolutionary process consists of three stages:

Decision making. All players make a decision whether to inoculate themselves against possible virus infections in the network. In case they choose to do so, they pay a cost , and pay nothing otherwise. Players’ propensity to inoculate is encoded in their strategy .

Virus propagation. After everyone has concurrently made a decision, expected costs of the nodes when the system is attacked by a virus are calculated. To do so, we use a process with
steps: in each step, the virus starts at a different node of the graph and spreads throughout the network. Inoculated players are unaffected by the virus and cannot transmit it to their neighbors, while unprotected players pay a cost once they become infected, and spread the virus to their other insecure neighbors. Uninfected players who are not inoculated do not pay anything. We will use the term ”realized cost vector” to describe the vector
, where infected nodes pay , inoculated nodes pay , and insecure but uninfected nodes pay nothing. Once the virus has swept over the system, infecting unprotected players and their unprotected neighbors, costs are recorded, infection status is reset, and the next virus starts at the next node, until every node has served as a starting point. Alternatively, the process can be randomized by letting sufficiently many viruses start at random nodes. Once this has happened, cumulative costs are averaged, giving expected negative payoffs for the current strategy configuration, and the next stage commences. 
Evolution of strategies. After receiving their expected negative payoff, players can assess the damage done and subsequently change their strategy based on comparisons with their neighbors’ payoffs, before they again decide which action to take, and the next game begins. It is important to realize that this updating process is based on purely local information: nodes only need to know their neighbors’ payoffs in order to make their decisions. This means that also the outcomes that can be realized may differ from the Nash equilibria that are found in the perfect information model of Aspnes et al. We consider dynamics that can take both selection and mutation into account: strategies typically evolve according to their (frequencydependent) fitness, but they can also randomly change with a small probability , which we will refer to as the mutation rate. This prevents absorbing states in the process and lets us compute a unique stationary distribution that gives the longterm frequencies of system configurations. In the limiting case of , the process always terminates and reaches a state where either all nodes are inoculated, or none are.
We differentiate here between two kinds of well known memoryless evolutionary dynamics, but in general we can configure our framework for general monotone imitation dynamics as described in [12]:

Genetic evolution: On one hand, we consider genetic evolution as described by the Moran process. In this context, we analyze two different variants: a deathbirth (DB) and birthdeath (BD) scenario, respectively (cf [44] and Algs. 1 and 2). In the DB scenario, a node is picked to die in each time step of the process. The vacancy is then filled by a copy of one of its neighbors, with the probability of one node being the replacement in some (possibly nonlinear) way proportional to its payoff, such that nodes with higher payoffs (or rather, fewer losses) have a higher chance of being chosen as the replacement. In the BD scenario, meanwhile, first a node is picked for reproduction in each round with probability proportional to its payoff. This node subsequently chooses one of its neighbors uniformly at random and replaces it with a copy of itself. After every update, payoffs are recomputed. To visualize an example of such a process, we illustrate the DB scenario in Fig. 1.

Cultural evolution: On the other hand, we also consider ”cultural” evolution through imitation dynamics in the form of the pairwise comparison process ([46] and Alg. 3). Here, a focal player picks a neighboring ”role model” node uniformly at random in every time step, observes its payoff, and switches to its strategy with probability
(3) where is the payoff of the neighbor and the node’s own payoff. This function is parameterized by the selection strength , which is a measure for the noise in the update probability, and with it, how much the payoff difference influences the dynamics. Thus, for , updating is random with probability , whereas for , strategies with lower costs are preferred.
These processes are simulated until timeout: once there have been update steps, we calculate the average welfare of the population (which is the average sum of payoffs), as well as the average count of how often the system visited the different states, and return.
With this dynamic game, we have a model that does not use the assumption of nodes having full information or memory; at most, they need to compare their payoff with the payoffs of their neighbors.
3.2 The evolutionary price of anarchy
In the analysis of an evolutionary game, a most fundamental question of interest is to which distribution different dynamics converge to, for various topologies and parameters. Such a stationary distribution of the Markov chain underlying the evolutionary dynamics contains the longrun frequencies of states, which correspond to the longrun probabilities of finding the system in the different states. Since we obtain an ergodic process on the space of all possible population compositions (hereon called configurations) by our elementary updating rules, this stationary distribution of inoculation states exists and is unique. It is also called the selectionmutation equilibrium x of a given evolutionary process, and forms the basis of further analysis. We note that it is the nonzero mutation rate that provides ergodicity – otherwise, the Markov chain would have absorbing states where all nodes are inoculated or none are.
We can subsequently find the average social cost for any of the dynamics we use, either by averaging over the total cost in each round (when the process is simulated), or multiplying x with the vector R containing the cost of all possible configurations, such that
(4) 
where , and is the average payoff of player in configuration . We measure the efficiency of a process by comparing the average social cost with the optimum . At this point, we introduce the concept of the evolutionary price of anarchy as the ratio of the average social cost (or payoffs) of a process against the social optimum. Similarly to the static PoA, we hence define, for a particular evolutionary dynamics and assuming negative payoffs:
(5) 
For positive payoffs, we define and note that in both cases, , as the static PoA. In general, this quantity gives an indication which processes are most conducive to spending large fractions of time in configurations that minimize overall cost (or maximize positive payoffs). We also note that in principle, the evolutionary PoA can be both smaller or larger than the static PoA.
Note that the concept of an evolutionary price of anarchy is neither bound to a particular game nor a particular evolutionary dynamics: it can serve as a general framework for analyzing a rich variety of dynamic games with arbitrary update dynamics.
4 Results and Analysis
In the following, we will consider pure strategies and the analysis of a setting with positive mutation rate . We will first show how to exactly calculate the selectionmutation equilibrium x of the evolutionary process for two fundamental graphs, and then use this to show how the ePoA can differ from a static analysis.
4.1 Analytical results
For simple graphs and pure strategies, , we can calculate the stationary distribution of the underlying Markov chain under the different dynamics. We consider two instructive cases here (similar cases have been studied also in [2]), situated at the opposite ends of the spectrum: the clique (which results in perfectly global information) and the star (which is fully local). In these cases, we find exact results.
4.1.1 Clique
In a clique, the states of the Markov chain are , denoting the number of currently inoculated nodes. Here, the threshold for infection of an arbitrary insecure node is 1, as an infected node automatically spreads the virus to all other not inoculated nodes. We use the entries of the cost vector as the realized negative payoffs of infected (), inoculated () and unaffected () nodes. For the expected payoffs of nodes using strategy , in the state , with either or , we then find the following simple expressions:
(6) 
and
(7) 
Meanwhile, the expressions for the transition probabilities including mutation (that is, with a mutation rate ) are as follows (for a derivation of Moran process transition probabilities, cf. e.g.[44]):
(8) 
and
(9) 
The terms generally depend on the dynamics being used. One caveat is that the virus inoculation game leads to expected payoffs . To be able to plug these terms into the equations for the Moran process probabilities, we use the standard assumption of an exponential fitness function (see [47]): expected payoffs are mapped to a fitness with the function , such that the fitness becomes
(10) 
We subsequently set the parameter , as is common in the literature. This quantity is now always positive, is larger for smaller costs (or equivalently, larger payoffs), and can be used in the standard Moran probabilities (cf. [36] and Eqns. 19 20 in the Appendix).
Meanwhile, for the pairwise comparison – imitation dynamics, we can still use the payoffs themselves without transforming them, and get Eqns. 21 and 22 (see Appendix).
From these transition probabilities, we can calculate the stationary distribution of the process: it is the normalized left eigenvector of the transition matrix, which is the tridiagonal matrix
, see Appendix section A.The mutationselection equilibrium is then the solution to
It gives the longrun frequencies of all possible states. These frequencies can also be obtained by simulating the process for long enough and counting how often each state occurs.
To be able to compare the evolutionary price of anarchy with the static price of anarchy, we first need to describe the Nash equilibria of the system. For the complete graph, by using Corollary 1.1, the static analysis predicts one equivalence class of Nash equilibria, , where exactly nodes are inoculated. We denote the efficiency of these equilibria as , the static price of anarchy.
In order to calculate , we first calculate the average social cost . We do so either by averaging over the total cost in each round in our simulations, or taking (cf. Eq. 4). For the complete graph, the vector containing the total system cost in all possible configurations, with inoculated nodes, has the components
(11) 
We also know the cost of the optimum; it is attained in the state with , which is the number of inoculated nodes where holds. The optimal cost is then . With this, we can use Eq. 5 to measure the efficiency of the different dynamics by finding their corresponding evolutionary price of anarchy as . We present our analysis below.
Using our evolutionary process, our findings can now be summarized in the following lemma:
Lemma 1.
For a fixed cost vector , large , any reasonable mutation rates , and intermediate to large selection strength (for the pairwise comparison process), we always recover the predicted equivalence class of Nash equilibria, , as the most abundant state in the selectionmutation equilibrium of both types of processes. That is, the process spends the largest fraction of time in the Nash equilibria, where exactly nodes are inoculated.
We note that there is also substantial weight on neighboring states of the Nash equilibria (with , where ) with worse total welfare, due to the stochasticity of the process. However, the average social cost is not substantially different from the cost of the Nash equilibria, since the weight on neighboring states is symmetrically distributed. The numerical results these insights are based on are provided via Figs. 2, 7 and 8, as well as upon request in full.
What this means for the evolutionary price of anarchy is expressed in the following corollary.
Corollary 2 (Evolutionary Price of Anarchy for Cliques.).
The evolutionary price of anarchy in a clique with nodes approaches the efficiency of the Nash equilibrium (the price of anarchy ), as and grows large, such that .
A straightforward argument shows why Lemma 1 and Corollary 2 hold. For the base case of the unstructured population on a complete graph, the perfect information setting corresponds to the local information setting, as each node only has distance 1 from every other node. Furthermore, the Markov chain underlying the stochastic evolutionary process is ergodic by , such that there exists a stationary distribution and the process converges to it. This stationary distribution then places most weight on the Nash equilibrium described in [2], as it is the configuration where nodes have no incentive to switch their strategy, and are aware of this fact, just as in the static case. The stochastic noise becomes smaller as the number of nodes grows larger (which inhibits the system spending too much time in the two extremal states of all nodes inoculated and all nodes insecure), and as the mutation rate becomes smaller (which sharpens the peak of the distribution at the equilibrium). This lets us recover the equilibrium results of [2].
4.1.2 Star graph
For star graphs , we can also numerically compute the expected payoffs and the Markov chain properties of the process. The states in this case are of the form , and . The parameter denotes the inoculation state of the center node, whereas gives the number of inoculated leaf nodes.
By using Corollary 1.1 again, we find two equivalence classes of Nash equilibria: class has inoculated leaf nodes (which in our notation is the state ), whereas contains the optimal equilibrium, which features the center as the only inoculated node (state ). We will show that for this highly structured population, the outcome can be quite different from the predictions of the static analysis of the oneshot game. Local evolutionary processes prevent the system from spending too much time in either of the equilibria classes. We detail this in the following paragraphs.
To see this, we first compute the expected payoffs of leaf nodes (, with ) and the center node () in the configurations :
(12) 
(13) 
and
(14) 
For the Moran and pairwise comparison – imitation dynamics we derive probabilities that describe the transitions without mutation, again with fitness . The exact expressions can be found in Section B.
We can now again get the transition matrix, with its entries (Eqns. 41 44 in the Appendix), and subsequently the selectionmutation equilibrium of the process with its corresponding average system cost.
Having calculated/simulated the stationary distribution, we observe the following (for results, see Appendix):

No matter the network size or the process, the Nash equilibria in , are not abundant states in the stationary distribution, with up to a factor in weight difference to the more frequent states. We instead find a high abundance of costly nonequilibrium states for some integers . There is also substantial weight on the beneficial configurations with low cost for the same values of .

The equilibrium is typically of far lower frequency than the nonequilibrium states . But it still plays more of a role overall than the optimum , which is a rare state at stationarity and almost never visited in the long run.
We will now argue why the process exhibits this offequilibrium behavior. First of all, starting from the above observations, it is straightforward to show why the optimum, that is, the state , cannot be an abundant state in the mutationselection equilibrium of a star graph.
Lemma 2 (The optimal Nash equilibrium is rare in the mutationselection equilibrium.).
Consider a star graph, with fixed but arbitrary number of nodes . For arbitrary mutation rates , arbitrary , and any of the three evolutionary processes we consider, the optimal Nash equilibrium cannot be an abundant state in the mutationselection equilibrium.
Proof.
In fact, the equilibrium is not even reached if not for mutation. To see this, consider the states and , and suppose . While both these states only differ in one node’s strategy from the Nash equilibrium, they cannot serve as starting points for a transition. State is absorbing in this scenario, as there is no inoculated node to pass on its strategy. Meanwhile, in the state , the one inoculated leaf node cannot change its strategy without the center node being not inoculated – the terms are always zero. This however leads to the opposite of the Nash equilibrium we are trying to reach. Thus, only a nonzero mutation rate can provide access to this state. At the same time, it is clear that the transitions from do not need mutation to reach the neighboring states and , which leads to a higher probability to leave the state than to reach it. This makes unsustainable in the long run. ∎
The following lemma states the corresponding result for the other Nash equilibria.
Lemma 3 (The Nash equilibria from the equivalence class cannot form an abundant state in the mutationselection equilibrium.).
Consider a star graph, with fixed but arbitrary number of nodes . For arbitrary mutation rates , arbitrary , and any of the three evolutionary processes we consider, the Nash equilibria of the form cannot be an abundant state in the mutationselection equilibrium. Instead, pairs of states of the form and , with the set depending on and the evolutionary dynamics used, act as sinks of the process.
Proof.
We use three facts arising from the transition matrix of the process. We argue here for the case of the pairwise comparison process, as it exhibits behavior that is less dependent on network size , but argumentation for the Moran BD scenario is very similar:

As the distance increases, the probabilities and decrease for , as it becomes more unlikely for an insecure node to pass on its strategy to a secure one. However, they do not decrease at the same rate. Rather, their ratio increases exponentially in , such that at some point it is far more likely for the center to become inoculated than for a leaf to switch to insecure. The ratio is
(15) It is 1 for the critical value of .

As increases, the probabilities and decrease at a constant rate, as it becomes overall less likely that an inoculated node will be chosen to pass on its strategy. This ratio is
(16) We see that is always larger than , as it is always more likely for one of the leaf nodes to get picked for reproduction if the center is inoculated.

As increases, the difference decreases and eventually changes sign. We find the ratios
(17) These ratios change from to at a particular value of , which is the breakeven point where moving forward in the lower level of the Markov chain with the center node inoculated is equally probable to moving in the opposite direction in the upper level of the chain.
Putting together these three facts, we end up with the following explanation for the behavior of the process.

Start in the Nash equilibrium .

Setting in Eq. (39), the random walk now has a times higher probability of leaving with a transition to the state than it does of leaving towards .

The probability of decreasing the number of inoculated leaf nodes in the states then gradually decreases with larger , and decreases faster than the probability of moving towards the state . This means that after , where the two probabilities are equal, a move to a state with the center being inoculated becomes more likely. In our example, , such that is the breakeven point, and the first state where is significantly different from is .

However, the probability of a transition from to only slowly increases with increasing . This leads to oscillations between opposite states and until the process escapes – and the probability of this happening is larger for the transition on the upper level from to , as long as the ratio in Eq. (43) .

The further away the process moves from , the longer it takes to escape from the oscillations between the pairs of states. This is most pronounced in the pairs (, ) corresponding to the threshold value of where moving forward in on the upper level and moving backward in the lower level are equal, as well as (, ). It is intuitive that the latter pair acts as the strongest sink, since the probabilities of leaving it in either way are smallest. Finding , the two pairs – and – are circled in Fig. 3, .

It is more likely that leaving the last sink happens via the transition . Then, the cycles repeat, starting from the other side. This makes it intuitive why the process will not find its way back easily to .
∎
The proof is additionally visualized by Fig. 3, where we show the case of the pairwise comparison process on a star graph with and .
Consider the process as a twodimensional random walk, defined by our transition probabilities (Eq. 23 40) in the limit of the mutation rate . Let ; the Nash equilibrium is then the state , as discussed above. In the example, we have , such that .
For easier readability, we use the notation for
(cf. Eqns 23 40). That is, gives the probability of moving one further step away from the Nash equilibrium by one leaf node switching to insecure; the probability of moving one step closer to by one leaf node switching to secure; the probability of the center switching to secure, and the probability of the center switching to insecure. The parameter can be thought to be the distance to . Note that this twodimensional random walk has a defined direction; there is no possibility to increase in the lower level of the chain (where the center is inoculated), and no possibility to decrease in the upper level (where the center is insecure).
We show in the Appendix that due to the setup of these transition probabilities, the random walk gets trapped in increasingly constricting cycles as it moves away from In the example of Fig. 3, these states form the set , with the most weight on and .
We thus have shown that using local information only, the system spends a high fraction of time in states that are not Nash equilibria, and will not reach the optimum. What does this mean for the evolutionary price of anarchy?
Seeing that the abundant states carry a high social cost compared to the optimum and also the worse equilibria in , it is already intuitive that will be larger than in the star graph, as long as the mutation rate is sufficiently small (). Indeed, using the stationary distribution to calculate the social cost as in Eq. 4, and then the evolutionary price as in Eq. 5, we find that the evolutionary process has to settle for a relatively high , and with it, a high (see Figs. 7 8). We summarize this in the following corollary:
Corollary 3 (Evolutionary Price of Anarchy for Star Graphs.).
For small mutation rates , arbitrary and arbitrary , the evolutionary price of anarchy in a star graph with nodes is at least equal to or higher than the static price of anarchy . That is, .
We note that the exact evolutionary price in relation to the static price of anarchy is determined by parameter choices. This means that with a mutation rate , for some choices of cost vector and network size, it is possible to achieve a slightly lower average cost than can be achieved by the worst possible solution to the oneshot game due to the higher mutation rate facilitating contributions from states that lead to a high total payoff (see Fig. 7).
However, our results let us conjecture that for a local information model with reasonably small mutation rates, we cannot hope to do much better on average than the worst Nash equilibrium in highly structured networks (like a star), much less reach the optimum, such that paying (at least) the price of anarchy is not only a theoretical possibility, but also a realized fact.
5 Simulation of more complex topologies
With the algorithms introduced above, we are able to simulate the process also for more complicated graphs. While numerical analysis is usually impossible in these cases  it is intuitive that obtaining numerically precise results on the stochastic process will be harder with increasing graph size and less inherent symmetry in the graph, as the resulting Markov chain can have as many as states already for pure strategies  we can always compute the average social welfare by simulating the different dynamics long enough. We can even feasibly find the stationary distribution for graphs that have some inherent symmetry (which prevents the Markov chain from being of size ). We will exemplify this by briefly discussing the evolutionary price of anarchy for two such graphs, which are highly symmetric and built from cliques and stars as the two basic components previously analyzed.
5.1 2clique network
An example of such a graph is depicted in Fig.5: two cliques, connected by a single path. The states of the Markov chain are now , with denoting the inoculation state of the two nodes where the cliques are joined (subsequently called the hubs), and denoting how many of the remaining nodes on each side are inoculated. For this graph, seeing that the underlying Markov chain of the process only has states, it is indeed possible to also find both the selectionmutation equilibrium and the average social cost by simulation. To be able to calculate , and compare it to the efficiencies of other equilibria as well as the static price of anarchy , we also compute the social welfare in all possible configurations (vector ), and find the optimum as the maximum over . This section describes our findings after running our simulations with for the network sizes and the realized cost vectors .

First considering static Nash equilibria as the baseline, we again use Corollary 1.1 with and find two equivalence classes of Nash equilibria. Class contains the states , with , which have both hubs insecure, and respective inoculated nodes in the remainders of the two cliques. The other class of equilibria, , is composed of the states and with , which are the states where at least one hub is always inoculated. In the case where is divisible by , the two equilibria classes are equivalent, giving the same cost. However, when is not divisible by , is more efficient with respect to the overall cost, such that it is the cost of the equilibria in which is used to calculate the .

In evolution, we find an for all three network sizes and all three evolutionary processes – the optimum is not an abundant state. It is also the case that , making the average cost slightly higher than the static price of anarchy, even though becomes smaller with increasing network size. In all the tested scenarios, the system does not spend a substantial fraction of time close to social optima and does not make up for partially costly offequilibrium behavior it exhibits otherwise. We again find that the Moran processes show behavior that is slightly dependent on the network size , whereas the pairwise comparison process gives more consistent results even for smaller , and also exhibits the smallest value of .
We have seen in Corollary 2 that for one clique on its own, we recover the Nash equilibria in our evolutionary process. However, the behavior of such a process on the 2CLIQUE graph leads to an outcome that differs both from the optimum and the static predictions for Nash equilibria. Intuitively, this is due to the link between the two hubs acting as a bottleneck for information flow.
5.2 2star network
Another example of a symmetric graph is depicted in Fig. 6: two stars, joined at their hubs. Here, our findings are the following:

We again begin by finding the static Nash equilibria and the static . In this case, there are three equivalence classes of equilibria: corresponds to its counterpart in the 2CLIQUE graph – it contains states (defined analogously to above), with , and is again the class with the most costly equilibria. It therefore is used for the static price of anarchy. Class consists of the states , with if and otherwise. Lastly, exists if , and features states of the type . Here, we can also explicitly characterize the optimum: it is always the state .

An evolutionary process on the 2STAR graph for the network sizes and ratios tested again leads to an . Seeing that the basic component of the graph – the simple star – already exhibits offequilibrium behavior, this is not too surprising. However, as opposed to the single star, we now observe that the Moran BirthDeath scenario is advantageous for all network sizes, as it leads to the lowest overall .
5.3 Cycle graph
Another structure of interest is the cycle graph. While this topology is simpler than the previously discussed 2star and 2clique networks, the process does not show straightforward behavior due to the large state space with few possible transitions. We start with a description of the states of the Markov chain and the Nash equilibria.
Encoding of states and size of the state space for network size .
Due to symmetry considerations, we can encode states corresponding to configurations by counting the number of insecure nodes in between every two inoculated nodes, and listing the number of gaps ordered by their size, starting at 1. As an example, we can take a network of size with inoculated nodes in the configuration , which corresponds to the state , where the first number in the list is simply . This is thereby also equivalent to the configurations , and . To enumerate and order these states, we can describe the sets of states with inoculated nodes more generally as integer partitions of , where an partition of is the set of partitions of with length . This way, we get a natural ordering on states  from the smallest partition length to the largest, and find a simple way to enumerate them. In the previous example, the corresponding partition of is . The total number of states in a network with nodes is then
(18) 
where is the partition function, giving the number of integer partitions of with length .
Nash equilibria.
For a state to be a Nash equilibrium, the largest insecure component cannot be larger than . For this condition to hold, the number of inoculated nodes has to be , and the Nash equilibrium states is then the partition of , where the largest part is at most . In our toy example above, using , those are the states and .
Results.
We run simulations for and , and again find the to be higher than the , making the cycle a suboptimal topology for efficient inoculation due to the fact that clusters of inoculated/insecure nodes can not be broken if not for mutation. Intuitively speaking, a Nash equilibrium with e.g. 2 inoculated nodes that are not next to each other will not easily be reached, as moving an inoculated node from point A to B has to involve mutation: starting from a cluster of two inoculated nodes, two imitation events and two mutation events are necessary to move the second node two steps away from the first one. This becomes more unlikely as . However, a surprising result can be seen in Fig. 7, for : the three evolutionary processes behave more or less the same in this case, and the efficiency is monotonic in the mutation rate, just as in the complete graph. An interesting direction for future work would thus be a more thorough analysis of the role of regular graphs in the behavior of stochastic evolutionary processes.
6 Related work
The term price of anarchy was introduced by Koutsoupias and Papadimitriou in [27], however, the idea of measuring inefficiency of equilibrium is older [8]. The PoA was first mainly studied for network models (see [35] for an overview), but literature now covers a wide spectrum, from health care to basketball [39]. The price of anarchy comes in different variations, also considering equilibria beyond Nash (e.g., the Price of Sinking [14]).
While in many text book examples, Nash equilibria are typically “bad” and highlight a failure of cooperation (e.g., Prisoner’s Dilemma, tragedy of the commons, etc.), research shows that in many application domains, even the worst gametheoretic equilibria are often fairly close to the optimal outcome [39]. However, these examples also have in common that users have full information about the game. While the study of incomplete information games also has a long tradition [16], much less is known today about the price of anarchy in games with partial information based on local interactions. In general, it is believed that similar extensions are challenging [39]. An interesting line of recent work initiated the study of BayesNash equilibria [41] in games of incomplete information (e.g., [41]), and in particular the BayesNash Price of Anarchy [39].
Another interesting concept is the stochastic price of anarchy [7], addressing the issue that some PoAs are sensitive to small perturbations, motivating the study of more stable notions. However, while the stochastic price of anarchy is also a dynamic concept, it requires even stronger assumptions on players: beyond current information (which is derived implicitly) the players also need knowledge of the execution history, to decide on their best action. Also the dynamic games based on learning dynamics [5], such as regret minimization [17] or fictitious play [18], require players to keep track of historical information. All these results hence do not apply to the evolutionary games we study here, as our strategies do not require the nodes to have memory, only local information about their neighbors’ payoffs. To the best of our knowledge, this paper is the first to consider the application of the price of anarchy to evolutionary games with this kind of limited information and memoryless players, also providing a novel perspective on the discussion of the impact of limited information in games.
In particular, we apply our framework in a case study, a specific network game where players interact topologically. Such games have already received much attention in the past: the price of anarchy was originally introduced with such games in mind [39]. However, such games are still not wellunderstood from an evolutionary perspective. In general, evolutionary games on graphs are considered challenging to analyze, even if the underlying topology is simple. See [1] for a review on this area.
More specifically, we revisited Aspnes et al.’s virus inoculation game [2] in this paper. Traditional virus propagation models studied virus infection in terms of birth and death rates of viruses [4, 11] as well as through local interactions on networks [25, 24, 48] (e.g., the Internet [37, 23]). An interesting study is due to Montanari and Saberi [33] who compare gametheoretic and epidemic models, investigating differences between the viral behavior of the spread of viruses, new technologies, and new political or social beliefs.
Aspnes et al.’s model is motivated by percolation theory [26] as infected node infect all unprotected neighbors, and already led to much followup work, e.g., also considering the impact of a small number of malicious players [34], or considering social network contexts where users account at least for their friends’ costs [31, 30]. There also exists literature on how to encourage highly connected nodes to inoculate [3]. However, we are not aware In this paper, we extended Aspnes et al.’s model accounting for local information and evolving players (the two main open questions stated in [2]), revisiting and refining their conclusions.
7 Conclusion
This paper introduced the evolutionary price of anarchy to study equilibrium behavior of simple agents interacting in a distributed system, based on local information. We showed that for memoryless agents, the resulting equlibria can be significantly different from their static equivalent, and that Nash equilibria are sometimes assumed only very infrequently.
We believe that our work can provide a novel perspective on the discussion of the impact of limited information in games. In particular, it opens several interesting avenues for future research. Our model still comes with several limitations due to the wellknown notorious difficulty of analyzing evolutionary multiplayer games on graphs. In particular, it will be interesting to analyze the ePoA for additional topologies and more general interaction models (beyond memoryless), as well as to explore randomized (mixed) strategies. It would also be interesting to prove or disprove our conjecture that processes based on imitation dynamics always result in a ePoA which is higher than the PoA in the virus inoculation game.
References
 [1] Benjamin Allen and Martin A Nowak. Games on graphs. EMS surveys in mathematical sciences, 1(1):113–151, 2014.
 [2] James Aspnes, Kevin Chang, and Aleksandr Yampolskiy. Inoculation strategies for victims of viruses and the sumofsquares partition problem. In Proc. 16th Annual ACMSIAM Symposium on Discrete Algorithms (SODA), pages 43–52, Philadelphia, PA, USA, 2005. Society for Industrial and Applied Mathematics. URL: http://dl.acm.org/citation.cfm?id=1070432.1070440.
 [3] James Aspnes, Joan Feigenbaum, Michael D Mitzenmacher, and David C Parkes. Towards better definitions and measures of internet security. 2011.
 [4] Norman TJ Bailey et al. The mathematical theory of infectious diseases and its applications. Charles Griffin & Company Ltd, 5a Crendon Street, High Wycombe, Bucks HP13 6LE., 1975.
 [5] Nicolo CesaBianchi and Gabor Lugosi. Prediction, learning, and games. Cambridge university press, 2006.
 [6] Krishnendu Chatterjee and Thomas A Henzinger. A survey of stochastic regular games. Journal of Computer and System Sciences, 78(2):394–413, 2012.
 [7] Christine Chung, Katrina Ligett, Kirk Pruhs, and Aaron Roth. The price of stochastic anarchy. In International Symposium on Algorithmic Game Theory, pages 303–314. Springer, 2008.
 [8] Pradeep Dubey. Inefficiency of nash equilibria. Mathematics of Operations Research, 11(1):1–8, 1986.
 [9] Alex Fabrikant, Ankur Luthra, Elitza Maneva, Christos H Papadimitriou, and Scott Shenker. On a network creation game. In Proceedings of the twentysecond annual symposium on Principles of distributed computing (PODC), pages 347–351. ACM, 2003.
 [10] Chris Fleizach, Michael Liljenstam, Per Johansson, Geoffrey M Voelker, and Andras Mehes. Can you infect me now?: malware propagation in mobile phone networks. In Proceedings of the 2007 ACM workshop on Recurring malcode, pages 61–68. ACM, 2007.
 [11] James C Frauenthal. Mathematical modeling in epidemiology. Springer Science & Business Media, 2012.
 [12] Drew Fudenberg and Lorens A. Imhof. Monotone imitation dynamics in large populations. Journal of Economic Theory, 140(1):229 – 245, 2008. URL: http://www.sciencedirect.com/science/article/pii/S0022053107001093, doi:https://doi.org/10.1016/j.jet.2007.08.002.
 [13] Drew Fudenberg, Martin A Nowak, Christine Taylor, and Lorens A Imhof. Evolutionary game dynamics in finite populations with strong selection and weak mutation. Theoretical population biology, 70(3):352–363, 2006.
 [14] Michel Goemans, Vahab Mirrokni, and Adrian Vetta. Sink equilibria and convergence. In Foundations of Computer Science, 2005. FOCS 2005. 46th Annual IEEE Symposium on, pages 142–151. IEEE, 2005.
 [15] Christophoros Hadjichrysanthou, Mark Broom, and Jan Rychtář. Evolutionary games on star graphs under various updating rules. Dynamic Games and Applications, 1(3):386, 2011.
 [16] John C Harsanyi. Games with incomplete information played by “bayesian” players, i–iii part i. the basic model. Management science, 14(3):159–182, 1967.
 [17] Sergiu Hart and Andreu MasColell. A simple adaptive procedure leading to correlated equilibrium. Econometrica, 68(5):1127–1150, 2000.
 [18] Josef Hofbauer and William H Sandholm. On the global convergence of stochastic fictitious play. Econometrica, 70(6):2265–2294, 2002.
 [19] Josef Hofbauer and Karl Sigmund. Adaptive dynamics and evolutionary stability. Applied Mathematics Letters, 3(4):75–79, 1990.
 [20] Josef Hofbauer and Karl Sigmund. Evolutionary games and population dynamics. Cambridge university press, 1998.
 [21] Rasmus IbsenJensen, Krishnendu Chatterjee, and Martin A. Nowak. Computational complexity of ecological and evolutionary spatial dynamics. Proceedings of the National Academy of Sciences, 112(51):15636–15641, 2015. URL: https://www.pnas.org/content/112/51/15636, arXiv:https://www.pnas.org/content/112/51/15636.full.pdf, doi:10.1073/pnas.1511366112.
 [22] Lorens A Imhof and Martin A Nowak. Stochastic evolutionary dynamics of direct reciprocity. Proceedings of the Royal Society of London B: Biological Sciences, 277(1680):463–468, 2010.
 [23] Jeffrey O Kephart and Steve R White. Directedgraph epidemiological models of computer viruses. In Computation: the micro and the macro view, pages 71–102. World Scientific, 1992.
 [24] Jeffrey O Kephart and Steve R White. Measuring and modeling computer virus prevalence. In Research in Security and Privacy, 1993. Proceedings., 1993 IEEE Computer Society Symposium on, pages 2–15. IEEE, 1993.
 [25] Jeffrey O Kephart, Steve R White, and David M Chess. Computers and epidemiology. IEEE Spectrum, 30(5):20–26, 1993.
 [26] Harry Kesten. Percolation theory for mathematicians, volume 423. Springer, 1982.
 [27] Elias Koutsoupias and Christos Papadimitriou. Worstcase equilibria. In Annual Symposium on Theoretical Aspects of Computer Science (STOC), pages 404–413. Springer, 1999.
 [28] Erez Lieberman, Christoph Hauert, and Martin A Nowak. Evolutionary dynamics on graphs. Nature, 433(7023):312, 2005.
 [29] Thomas Locher, Patrick Moor, Stefan Schmid, and Roger Wattenhofer. Free riding in bittorrent is cheap. In Proc. 5th Workshop on Hot Topics in Networks (HotNets), pages 85–90, November 2006.
 [30] Dominic Meier, Yvonne Anne Oswald, Stefan Schmid, and Roger Wattenhofer. On the windfall of friendship: Inoculation strategies on social networks. In Proc. 9th ACM Conference on Electronic Commerce (EC), pages 294–301, 2008.
 [31] Dominic Meier, Yvonne Anne Oswald, Stefan Schmid, and Roger Wattenhofer. On the windfall and price of friendship: Inoculation strategies on social networks. Elsevier Computer Networks (COMNET), 2014.
 [32] Swarup Mohalik and Igor Walukiewicz. Distributed games. In Paritosh K. Pandya and Jaikumar Radhakrishnan, editors, FST TCS 2003: Foundations of Software Technology and Theoretical Computer Science, pages 338–351, Berlin, Heidelberg, 2003. Springer Berlin Heidelberg.
 [33] Andrea Montanari and Amin Saberi. Convergence to equilibrium in local interaction games. In 2009 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pages 303–312. IEEE, 2009.
 [34] Thomas Moscibroda, Stefan Schmid, and Roger Wattenhofer. When selfish meets evil: Byzantine players in a virus inoculation game. In Proc. 25th Annual Symposium on Principles of Distributed Computing (PODC), pages 35–44, July 2006.
 [35] Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V Vazirani. Algorithmic game theory. Cambridge University Press, 2007.
 [36] Martin A. Nowak. Evolutionary Dynamics. Harvard University Press, 2006. Chapter 8.
 [37] Romualdo PastorSatorras and Alessandro Vespignani. Epidemics and immunization in scalefree networks. arXiv preprint condmat/0205260, 2002.
 [38] Amir Pneuli and Roni Rosner. Distributed reactive systems are hard to synthesize. In Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science (FOCS), pages 746–757. IEEE, 1990.
 [39] Tim Roughgarden. The price of anarchy in games of incomplete information. ACM Transactions on Economics and Computation (TEAC), 3(1):6, 2015.
 [40] Tim Roughgarden and Éva Tardos. How bad is selfish routing? Journal of the ACM (JACM), 49(2):236–259, 2002.
 [41] Satinder Singh, Vishal Soni, and Michael Wellman. Computing approximate bayesnash equilibria in treegames of incomplete information. In Proc. 5th ACM conference on Electronic Commerce (EC), pages 81–90. ACM, 2004.
 [42] J Maynard Smith and George R Price. The logic of animal conflict. Nature, 246(5427):15, 1973.
 [43] John Maynard Smith. Evolution and the theory of games. In Did Darwin Get It Right?, pages 202–215. Springer, 1988.
 [44] Christine Taylor, Drew Fudenberg, Akira Sasaki, and Martin A Nowak. Evolutionary game dynamics in finite populations. Bulletin of mathematical biology, 66(6):1621–1644, 2004.
 [45] Arne Traulsen and Christoph Hauert. Stochastic evolutionary game dynamics. Reviews of nonlinear dynamics and complexity, 2:25–61, 2009.
 [46] Arne Traulsen, Martin A. Nowak, and Jorge M. Pacheco. Stochastic dynamics of invasion and fixation. Phys. Rev. E, 74:011909, Jul 2006. URL: https://link.aps.org/doi/10.1103/PhysRevE.74.011909, doi:10.1103/PhysRevE.74.011909.
 [47] Arne Traulsen, Noam Shoresh, and Martin A Nowak. Analytical results for individual and group selection of any intensity. Bulletin of mathematical biology, 70(5):1410, 2008.
 [48] Chenxi Wang, John C Knight, and Matthew C Elder. On computer viral infection and the effect of immunization. In Computer Security Applications, 2000. ACSAC’00. 16th Annual Conference, pages 246–256. IEEE, 2000.
 [49] Jörgen W Weibull. Evolutionary game theory. MIT press, 1997.
Appendix A Clique: Markov chain transition probabilities
The standard Moran probabilities for the complete graph (clique) look the following:
(19) 
and
(20) 
The pairwise comparison process does not use exponential fitness, but the original payoffs:
(21) 
and
(22) 
The transition matrix for the process with mutation looks the following:
The terms correspond to the terms in Eqns. 8f.
In the version of the complete graph that corresponds to the standard case of a wellmixed population (where we have a positive probability of the same node being chosen for reproduction and death), the two Moran processes are equivalent. We note that while Eqns. 19 20 correspond to the standard Moran process on complete graphs as described in the literature [36], it is however simple to change the expressions such that there is no possibility of selfreplacement for the nodes. This does not change our qualitative results, but does introduce a difference between birthdeath and deathbirth processes. We remark that the difference between and is always slightly smaller for the pairwise comparison process as compared to the Moran processes. For large values of , both kinds of stochastic processes lead to an outcome with the same average efficiency as the Nash equilibria. Meanwhile, for smaller , the outcomes are somewhat different on first glance: the Moran process has the system drift towards the two extremal configurations, where either all nodes are inoculated or none are. This comes with an overall increased social cost, and is due to the inherent randomness of the stochastic process. This effect can then be mitigated by using the pairwise comparison process with intermediate to strong selection strength instead, or increasing the parameter in the transformation of payoffs into fitness. This lets us again recover the Nash equilibria as the most abundant states, and with them the corresponding efficiency.
Appendix B Star graph transition probabilities
It is intuitive that the two Moran processes on the star graph will not lead to the same dynamics, unlike in the case of the complete graph: for the clique, the global minimal cost and the minimal cost of a neighbor coincide, giving equal probabilities for the best node overall to reproduce and for the best neighboring node to reproduce. This does not hold for the star graph, such that the two Moran processes are no longer equivalent (cf.[15]. The probabilities are the following:
Deathbirth process
(23) 
(24) 
(25) 
(26) 
(27) 
(28) 
Birthdeath process
(29) 
(30) 
(31) 
(32) 
(33) 
(34) 
Pairwise comparison – imitation process
(35) 
(36) 
(37) 
(38) 
(39) 
(40) 
The transition matrix has the entries
(41) 
(42) 
(43) 
(44) 
Comparing the probabilities for the two Moran processes now, we can indeed see that they will not lead to the same dynamics (e.g. by comparing the two expressions for ). In fact, we make the following observations:

The different evolutionary dynamics indeed show different behavior, which is reflected in the weight distribution on the states as given by their invariant distribution, as well as their evolutionary price of anarchy. We find that for any choice of parameters , and , the Moran DeathBirth process leads to the worst of the three described processes. It spends a large fraction of time in the two extremal states (for small mutation rates ), or has an almost uniform invariant distribution over the states.

For the other two processes, Moran BirthDeath and pairwise comparison, we also observe offequilibrium behavior. The outcome of the BirthDeath scenario again depends on the network size : for small , there is again drift towards the two extremal states and as a result of random noise. The smaller the mutation rate, the more pronounced this drift is, and the larger the network size needed to balance it out. The pairwise comparison process however does not show the same strong dependence on the network size in its behavior.
Comments
There are no comments yet.