I Introduction
Evolutionary game theory is a mathematical framework for investigating the dynamics of strategies in populations ranging from two players to structured societies
Nowak (2006). Trust between players plays a major role in the evolution of games in a social context, and has deep implications for the collective action of social and human systems Reina and Reina (2006); Grodzinsky et al. (2015); Petraki and Abbass (2014). As players rely on trust to handle complex problems and make decisions, relationships are formed. These relationships, in turn, give rise to opportunities and allow trust to spread Abbass et al. (2016). A recent study found that players are more trustworthiness when having uncalculating cooperation Jordan et al. (2016).In terms of models, the most wellknown version of trust games considers interactions between two types of players: investors (or trusters) and trustees Berg et al. (1995); Cox (2004); KingCasas et al. (2005); Tarnita (2015). In these games, the investor must first decide whether to trust the trustee. If the decision is positive, the trustee must then decide whether to be trustworthy or not. Although twoplayer games are the most common configuration for trust games in the literature, they have their limitations. For instance, we cannot generalize some of the insights found in pairwise interactions to games with multiple players having more than two strategies Gokhale and Traulsen (2010).
Abbass et al. Abbass et al. (2016) therefore proposed an evolutionary Nplayer trust game to generalize the concept of trust to a wellmixed population of individuals who play a trust game concurrently. Chica et al. Chica et al. (2017) subsequently showed that promoting trust in this Nplayer trust game depends on the social network structure. In their model, a structured population of players have three possible strategies to choose from: to be an investor (), a trustworthy trustee (), or an untrustworthy trustee (). Players interact with their direct neighbors on the social network, and obtain their payoff values repeatedly through a number of time steps. They evolve their strategies according to the obtained payoffs and update their strategies in a nondeterministic way (i.e., using a proportional imitation update rule).
In this paper, we extend on the work of Chica et al. Chica et al. (2017) by studying the implications in terms of spatiotemporal correlations of different update rules in the same networked multiplayer trust game. In addition to a proportional imitation update rule, we also consider unconditional imitation, a local Moran process, and a hybrid approach using unconditional imitation and a voter model in a stochastic function. In threestrategy games, systems can exhibit repetitive succession of oscillatory and stationary states Szolnoki et al. (2009). This is because when having three strategies, the outcome can be considerably more complicated and the game can end in everincreasing oscillations or with the elimination of some of the existing strategies Nowak (2006). It is thus important for us to examine the effects of different update rules and how the system dynamics would change in this context.
In our simulations, we consider different social network topologies (i.e., a simple regular lattice Nowak and May (1992) and scalefree (SF) networks Albert and Barabási (2002); Barabási and Albert (1999)). We also explore the effects of the update rules for different values of the temptation to defect ratio in the trust game. This ratio defines the temptation to defect investors’ trust by the trustees and measures the level of difficulty of the game. We evaluate the final steady state (i.e., surviving strategies) and global net wealth (i.e., total payoffs) of the population to observe which update rules and under which conditions players can better promote trust and obtain the highest net wealth of the population.
The effects of the different update rules are then examined to better understand how trustworthy and untrustworthy behaviors emerge and what is their corresponding global net wealth like. The dynamics of the model are first studied from the spatial perspective. It is known that spatial games of evolution can generate dynamic fractals, gliders and kaleidoscopes Nowak and May (1992); Nowak (2006), and so we analyze the spatial formations as well as those spatial correlations between the players’ behaviors and strategies. Finally, we show the temporal dynamics of the model by analyzing the signal of the system and investigate if the majority of the rules generate those correlations. We also explore if more elaborated correlations with both spatial and temporal types occur in the model’s output.
Ii Model
ii.1 Model description
The multiplayer trust game consists of a finite set of agents occupying the nodes of a social network, with edges denoting interactions between them (both for accumulating payoffs and strategy updating Nowak et al. (2010)). Each player chooses a strategy from three possibilities at every time step () Chica et al. (2017): being an investor (strategy ), being a trustworthy trustee (strategy ), and being an untrustworthy trustee (strategy ). During the evolutionary game process, all players interact with their directly connected neighbors at the same time in a single game (group interaction Chiong and Kirley (2012)).
Initially, the agents’ strategies are assigned at random. Let us consider a local neighborhood with a focal agent, , as shown in Figure 1. The game is played between and all its direct neighbors. In the neighborhood, there are investors, trustworthy trustees, and untrustworthy trustees. The equality must always be fulfilled for consistency’s sake, where is the degree (number of connections) of focal node .
By the rules of the game, every investor releases a unit of payoff per time step. Since all the agents in the neighborhood are playing the game as a group, this means there are a total of units. Each of the trustees receives an equal division of this total quantity: . A trustworthy trustee (strategy ) then returns the received fund multiplied by to the investors: (i.e., to each of the investors, where ). Each of the trustworthy trustees also receives the same amount of fund as the investors. That is, the investment produces in total . On the other hand, untrustworthy trustees (strategy ) return nothing but keep for themselves the received fund multiplied by : .
The parameter acts as the temptation to defect ratio (to be untrustworthy), defined by , and it is used to control the difficulty level of the trust game. values must fulfill the restriction . Taking into account the number of players in the group and , we can determine the net wealth of individual agents, based on their payoffs, and according to the strategy adopted by themselves and their direct neighbors. The net wealth of a focal player is obtained as follows:
(1) 
where , , and are the numbers of investors, trustworthy trustees, and both trustworthy and untrustworthy trustees in the local neighborhood of , including the focal agent itself. The net wealth of focal agent is 0 when there is no trustee in the neighborhood (i.e., when ). We are interested in the global net wealth of the population , calculated as . After playing a game and calculating their payoffs, the agents have an opportunity to update their strategies according to the payoffs received. This operation is carried out according to one of the evolutionary update rules, which will be described next.
ii.2 Update rules
Agents decide which strategy to choose based on the population strategies in the previous time step (i.e., ) and their payoff Chica and Rand (2017). The strategy update follows an evolutionary procedure based on neighbor imitation that can be interpreted as information exchange within a social learning process Nowak et al. (2010). To be more precise, at timestep , a focal agent (independently from its strategy) evaluates its previous payoff results in and decides whether to imitate a direct neighbor ’s strategy or not by using an evolutionary update rule. These rules of imitative nature are widely employed in the literature and represent a situation where bounded rationality or lack of information forces players to copy (imitate) others’ strategies Schlag (1998). In this work, we have considered the following four update rules because of their different features (i.e., some are deterministic and others stochastic; some of them evaluate all the neighbors and can make mistake while others not):

Unconditional imitation (UI) Nowak and May (1992). A player directly copies the strategy of the best performing neighbor only if the payoff is better than . UI is a deterministic and strategic rule based only on payoff maximization when imitating others.

Hybridization of UI and a voter model (UIVM). By using this mechanism, players imitate others by hybridizing two rules: UI, a totally deterministic and payoff based rule, and the voter model Holley and Liggett (1975), a random decisionmaking mechanism based on just picking the strategy of one of the neighbors, , at random. As done in Vilone et al. (2014)
, our model uses a probability parameter
for players to select between the two update rules: the voter model is chosen as the update rule by player with probability while UI is chosen by player with probability . We set for our experiments after some preliminary analysis across a range of other values. 
The Moran (MO) process Moran et al. (1962). With this update rule, focal agent evaluates all its neighbors’ payoffs and assigns probabilities to them. This update rule is also stochastic but based on imitation dependent on the payoffs. At each time step there are possibilities to imitate each of the neighbors. Player thus can make mistakes by imitating a worseperforming neighbor in terms of payoffs.

Proportional imitation (PROP) Helbing (1992). It is a pairwise and stochastic update rule where players cannot make mistakes during the imitation process (i.e., never imitate players with a lower payoff value). Agent first evaluates if the individual net wealth of neighbor in the previous time step () is higher than its own (). If the net wealth of is higher, will adopt the strategy of with a probability depending on the difference between their payoffs:
(2) where is the maximum payoff distance between two players to have properly normalized. The minimum possible net wealth for the game is when the focal agent is an investor and its neighborhood is formed by all untrustworthy trustees. The maximum possible net wealth for the game occurs when the focal agent is untrustworthy and all its neighbors are investors. In this case, is equal to .
Iii Global Net Wealth
First, we will analyze the results of global net wealth of the population (i.e., the conditions and update rules to promote the highest sum of payoffs of all players of the population). The simulations were run for time steps until the model reached a stationary state using a population of 1,024 players. We set to 6, as in Abbass et al. (2016); Chica et al. (2017). The temptation to defect ratio was set to three different values: and , which cover three different trust scenarios: from an “easier” to a “harder” evolutionary trust game. All experiments were repeated for independent Monte Carlo (MC) realizations.
We performed sensitivity analysis on the initial population conditions where the global net wealth was averaged over the last of the steps. Figure 2 shows heatmaps of the results for the four update rules (individual columns of the panel) and three values with two social network topologies (see the six rows of the panel). Specifically, we show results for a regular lattice (REGULAR) and a SF network with average degree , generated using the BarabasiAlbert algorithm. The first three rows report on the net wealth using SF, with the game level rises from easy to a more difficult level (). The last three rows report on the same results when considering a regular lattice for the same difficulty game conditions.
We can see how, even for hard game conditions (), both UI and UIVM can promote high “cooperation” levels under a wide range of initial population conditions when using a regular lattice (see the bottom left plots of the panel). Generally, both UIbased update rules (first two columns of the panel) perform equally in terms of promoting trust cooperation. However, when using a regular lattice, the use of UIVM can generate a wider region of high global net wealth than when using the classical UI.
Trust cooperation results obtained by UIbased rules, however, are not as good as MO and PROP update rules in those scenarios where we used a SF topology under easy game conditions (the first row of Figure 2). The MO rule is able to obtain higher global net wealth values but under narrower initial conditions (see the heatmap plot at position (1,3) of Figure 2). When the game becomes harder, both MO and PROP update rules cannot obtain high net wealth values when using a SF topology and trust cannot be promoted (same in the case of UIbased rules). See for example those heatmaps in the third row of the panel, which correspond to the case of . Both MO and PROP update rules are not able to promote nonzero global net wealth at the end of the simulations.
Iv Model Dynamics
In this section, we focus on the spatial and temporal dynamics of the game using different update rules. For all the analyses, we ran 100 independent MC realizations with an initial population formed by of players with strategy , with strategy , and with strategy . We considered regular lattices of sizes (i.e., 1,024 players) and (i.e., 14,400 players) to better understand the spatial correlations between them.
iv.1 Spatial effects
We start by showing spatial snapshots of the lattices after reaching an equilibrium state. Figure 3 shows, for each of the four update rules and three values, the spatial layout of the lattices with the three possible strategies. Nodes in blue are those with strategy , nodes in green are those with strategy , and nodes in red are those with strategy . We also show the spatial layout when considering the lattice (14,400 player) in Figure 4.
For both lattice sizes we can see similar cluster and spatial formations. Also, the existing playing strategies are in line with the global net wealth observed in the previous section and Figure 2. UI and UIVM rules, even with hard cooperation conditions, are able to eliminate all the players (only blue and green cells remain). With , however, untrustworthy players totally dominate the population when the MO rule is in place, and partially dominate the population when the PROP rule is used.
The spatial layout also suggests a fractal structure. This is especially clear when considering the UI rule, which provokes the emergence of “organic” clusters in the lattice. In order to further investigate the fractal properties of the game, we study, in a steadystate stage of the simulation, the relation of values (i.e., number of investors) when increasing a box size within the lattice. Figure 5 shows this relationship for both lattice sizes. Again, in this analysis we see that the number of players (i.e., the size of the lattice) does not affect the fractal properties of the game.
We have also fitted the exponents of these curves to find the fractal dimensions (i.e., the number of players playing each of the three strategies) and found their exponential values for a function. These values are shown in Figure 6 for values ranging from to on the X axis. The right plot of the same figure shows the number of existing playing strategies to complement the fractal analysis. As we can observe in these plots, for all the update rules, the output of the spatial model presents a fractal structure with values between and . This fractal structure is constant under different game difficulty conditions and for the three dimensions, although clearer for the number of investors .
The last spatial correlation analysis of this section consists of calculating the spatial correlation coefficients for all the players in the lattice located at a distance . We follow a similar approach in FernándezGracia et al. (2014) by computing a spatial correlation function as follows:
(3) 
where is the number of pairs of agents at distance in the lattice, is the distance in the lattice between players and , and represents the three possible strategies of player . Figure 7 shows all the values for both and lattices for the four update rules. We find similar trends for both lattice dimensions (please note the X axis is relative to the lattice size), which again corroborates similar model behavior independent of the number of players. We observe cyclic oscillatory behavior when using one of the two UIbased rules (top plots of the figure).
iv.2 Temporal correlations
In this section, we explore temporal correlations of the model. Figure 8 shows, for each update rule, the power spectrum of the signal, which refers to the total number of players playing strategy (i.e., ) at each step of the simulation. This signal is obtained when the simulation reaches a steady state. The lefthand plot shows the power spectrum for the UI rule in a linear scale. We see that the signal has a high frequency of , which means that there is a complete temporal correlation with a period of 2 steps. This is because of the oscillatory behavior of UI within the model, also shown in the spatial analysis in the next section.
More interestingly, the plot on the right shows the power spectrum of the other three update rules in a loglog scale. The three spectra of the rules have a powerlaw distribution. This distribution is especially clear for the PROP update rule, which shows a linear drop. These powerlaw distributions of the spectra of the rules show that the model’s output has many different low frequencies. Therefore, when using nondeterministic update rules, the model has a longterm memory (i.e., long temporal correlations). This behavior is not common in this kind of models, as many of them exhibit stochastic dynamics, where there is only a short memory and the decision does not depend on previous history Torney et al. (2011). Finally, we also see how the power spectrum of the UIVM rule (blue line) has a highfrequency peak because of the deterministic nature of UI.
iv.3 Spatiotemporal correlations
We have calculated the spatiotemporal correlations among pairs of players in the lattice as the final stage of our experimentation analysis. Figure 9 shows plots with the timelag Pearson correlation values of four players’ time series with respect to a focal player. The calculation process was done as follows. We first set a focal agent at random and four additional players situated at a distance of 2, 4, 6, and 10 in the lattice with reference to the focal one. The time series strategy evolution of the focal agent and the other four players in the last 50% of the steps are obtained. Later, the focal agent time series is compared to the other four players’ time series to find spatiotemporal correlations.
Here, we can see similar behavior to that observed when analyzing the spatial and temporal correlations independently in the previous sections. There are high correlation values for the UI deterministic rules (both UI and UIVM) and these correlations are cyclic and for all the considered distances. This is related to the oscillatory behavior seen for the UI rule. In the case of the other two update rules (please note a lower Y scale for the MO and PROP plots), correlation values are lower. Proportional imitation (the bottom right plot of Figure 9) has the highest correlation with the closest players (i.e., at distance 2). When using the MO update rule, correlations tend to become a null value when increasing the lag time between the series. As happened with the PROP rule, there is a big difference between the correlation values of closer players and those at 6 or 10 nodes, where there is not any spatiotemporal correlation.
V Discussion
In this work, we have shown the dynamics of different update rules for a multiplayer trust game of three strategies. First, we looked at which rules and social network topologies can obtain the highest trust cooperation among players in the population. Based on the simulation results, we can divide the update rules into two groups: (1) the UI strategy and its hybridized update rule with a voter model; and (2) the PROP and MO rules. PROP and especially the MO update rule obtain the best results in terms of global trust promotion when the game is easy (), and the network topology is SF. When the game is difficult, however, both UIbased rules are able to better promote trust and obtain higher global net wealth values for very different initial population conditions. In contrast, MO and PROP are not able to promote trust under difficult game conditions. With respect to the network topology, using a regular lattice enables better trust promotion leading to higher global net wealth.
Apart from evaluating trust promotion by the rules, we studied the spatial and temporal model dynamics. We found that the UIbased rules form spatial clusters of spongelike areas. These patterns are created to enable the maximization of groups of trustworthy trustees (strategy ) with at least one investor (strategy ). More interestingly, we found that not only UI but all the update rules generate a fractal space on the lattice when reaching a steadystate stage. Generally, spatial correlations are important and occur mainly in closer players within the lattice. By studying the power spectrum of the model’s output we also observed that when introducing stochasticity in the rules, a longterm memory appears in the system. This means that for the UIVM, MO, and PROP rules, there are signals at low frequencies and their spectra are of powerlaw nature.
Another interesting behavior is that the system is not frozen when reaching a steady state for deterministic rules such as the UI rule. There is a phenomenon related to “switching” or cyclic oscillatory behavior (e.g., from investor strategy to trustworthy trustee or the other way round). From the temporal and spatiotemporal analyses we found these oscillations and maximum correlations with a 2step period (a high frequency of ). It is also clear that all of these switching nodes must have more than one connection to create a “chain” of switching nodes. The latter switching behavior was not observed in the PROP update rule. We think the reason for not having this behavior when using the PROP rule is because it is the only rule that only evaluates one neighbor to make a decision. When considering rules such as UI, UIVM, and MO, if one neighbor changes the strategy, the decision of focal player changes totally, as all the neighbors are evaluated to make a decision to imitate.
These important findings about the hidden dynamics and spatiotemporal correlations in the game show the complexity of managing trust in social networks in the long run. They also open doors for future research on evolutionary game theory in general and evolutionary trust games in particular.
References
 Nowak (2006) M. A. Nowak, Science 314, 1560 (2006).
 Reina and Reina (2006) D. S. Reina and M. L. Reina, Trust & betrayal in the workplace: Building effective relationships in your organization (BerrettKoehler Publishers, 2006).
 Grodzinsky et al. (2015) F. S. Grodzinsky, K. W. Miller, and M. J. Wolf, Philosophy & Technology 28, 91 (2015).
 Petraki and Abbass (2014) E. Petraki and H. A. Abbass, in the 2014 Seventh IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA) (IEEE, 2014) pp. 1–7.

Abbass et al. (2016)
H. Abbass, G. Greenwood, and E. Petraki, IEEE Transactions on Evolutionary Computation
20, 470 (2016).  Jordan et al. (2016) J. J. Jordan, M. Hoffman, M. A. Nowak, and D. G. Rand, Proceedings of the National Academy of Sciences 113, 8658 (2016).
 Berg et al. (1995) J. Berg, J. Dickhaut, and K. McCabe, Games and Economic Behavior 10, 122 (1995).
 Cox (2004) J. C. Cox, Games and Economic Behavior 46, 260 (2004).
 KingCasas et al. (2005) B. KingCasas, D. Tomlin, C. Anen, C. F. Camerer, S. R. Quartz, and P. R. Montague, Science 308, 78 (2005).
 Tarnita (2015) C. E. Tarnita, Games 6, 214 (2015).
 Gokhale and Traulsen (2010) C. S. Gokhale and A. Traulsen, Proceedings of the National Academy of Sciences 107, 5500 (2010).
 Chica et al. (2017) M. Chica, R. Chiong, M. Kirley, and H. Ishibuchi, IEEE Transactions on Evolutionary Computation (in press) xx, x (2017).
 Szolnoki et al. (2009) A. Szolnoki, M. Perc, and G. Szabó, Physical Review E 80, 056104 (2009).
 Nowak and May (1992) M. A. Nowak and R. M. May, Nature 359, 826 (1992).
 Albert and Barabási (2002) R. Albert and A.L. Barabási, Reviews of Modern Physics 74, 47 (2002).
 Barabási and Albert (1999) A. L. Barabási and R. Albert, Science 286, 509 (1999).
 Nowak et al. (2010) M. A. Nowak, C. E. Tarnita, and T. Antal, Philosophical Transactions of the Royal Society of London B: Biological Sciences 365, 19 (2010).
 Chiong and Kirley (2012) R. Chiong and M. Kirley, IEEE Transactions on Evolutionary Computation 16, 537 (2012).
 Chica and Rand (2017) M. Chica and W. Rand, Journal of Marketing Research 54, 752 (2017).
 Schlag (1998) K. H. Schlag, Journal of Economic Theory 78, 130 (1998).
 Holley and Liggett (1975) R. A. Holley and T. M. Liggett, The Annals of Probability , 643 (1975).
 Vilone et al. (2014) D. Vilone, J. J. Ramasco, A. Sánchez, and M. San Miguel, Physical Review E 90, 022810 (2014).
 Moran et al. (1962) P. A. P. Moran et al., The Statistical Processes of Evolutionary Theory (1962).
 Helbing (1992) D. Helbing, Physica A 181, 29 (1992).
 FernándezGracia et al. (2014) J. FernándezGracia, K. Suchecki, J. J. Ramasco, M. San Miguel, and V. M. Eguíluz, Physical Review Letters 112, 158701 (2014).
 Torney et al. (2011) C. J. Torney, A. Berdahl, and I. D. Couzin, PLoS Computational Biology 7, e1002194 (2011).
Comments
There are no comments yet.