In a binary () coordination game over a graph, where initially all agents are in the Nash equilibrium , what is the minimum number of agents that if forced to will push the system to converge to the Nash equilibrium of all ’s under a best response dynamics? This paper is devoted to the analysis of this problem and to the design of an algorithm for an efficient solution.
The problem considered can be framed in the more general setting of studying minimal interventions strategies needed to make a multi-agent system governed by agents’ myopic utility maximization, to drive from a Nash equilibrium to a desired another one. In game theory, typically, interventions have been modeled as perturbations of the utility functions (e.g. taxes and prices in economic models or tolls in transportation systems). Here we instead take a different viewpoint: that of individuating a subset of nodes (hopefully small) that if suitably controlled will lead the entire system to the desired configuration. The minimum cardinality of this set can also be interpreted as a measure of resilience of the system: the larger it is, the more difficult is for an external shock to destabilize it.
The problem of determining the best set of nodes to exert the most effective control in a networked system has recently appeared in other contexts: for instance in Yildiz et al. (2013) and Vassio et al. (2014) authors study the problem of the optimal position of stubborn influencers in linear opinion dynamics.
Binary coordination games have received a great attention in the recent years as one of the basic models for games with strategic complementarities Jackson and Zenou (2015). Its variegate applications include modeling of social and economic behaviors like adopt a new technology, participate in an event, provide a public good effort.
This game is analyzed in detail in Morris (2000) where the key concept of cohesiveness of a set of players is introduced and then used in characterizing all NE’s. Moreover, the question if an initial seed of influenced players (that maintain action in all circumstances) is capable of propagating to the all network is addressed in the same paper and an equivalent characterization of this spreading phenomenon is also expressed in terms of cohesiveness. This contagion phenomenon is exactly what we want to analyze: subset of nodes from which propagation is successful will be called sufficient control sets and our goal is to find such sets of minimum possible cardinality.
The condition proposed in Morris (2000) is computationally quite demanding and can not be used to directly solve our optimization problem. Indeed, even to determine if a single set is a sufficient control set, it requires a number of check growing exponentially in the cardinality of the complement of such set.
The complementary problem of understanding, given an integer , what is the maximum possible spreading of the state , starting from an initial seed of influenced players, was studied in a seminal paper by Kempe et al. (2003). While their problem and ours are related, they are independent, in the sense that solving one does not provide a solution of the other. Another point worth stressing is that, in their setting, Kempe et al. (2003) consider agents equipped with random independent activation thresholds and take as functional to be optimized the average size of the maximum spreading. They prove that such functional is sub-modular and then they design a greedy algorithm for obtaining suboptimal solutions. The randomness that they introduce is actually crucial in their approach, as the functional considered would not be sub-modular for deterministic choices of thresholds.
In this paper we consider a scenario when all agents have a fixed threshold , thus not covered in Kempe et al. (2003), and we design an iterative search randomized algorithm with provable properties of convergence towards sufficient control sets of minimum cardinality. The core of the algorithm is a time-reversible Markov chains over the family of all sufficient control sets that starts with the full set, moves through all of them in an ergodic way, and concentrates its mass on those of minimum cardinality .
We conclude this introduction with a brief outline of the paper. In the final part of this section we report some basic notation used throughout the paper. Section 2 is dedicated to the formal introduction of the problem. The main technical parts are Sections 3 and 4. In Section 3, we introduce the important notion of monotone crusade (appeared for other purposes in Drakopoulos et al. (2014, 2016)) and we give an equivalent (but more operative) characterization of sufficient control sets. In Section 4
we introduce a family of reversible Markov chains whose invariant probability is proven to concentrate on the optimal sufficient control sets. Section5 describes the algorithm, based on the Markov chains introduced in the previous section, and presents some simulation results. Finally, Section 6 ends the paper.
Vectors are indicated in bold-face letters . For are two vectors of the same dimension, the notation indicates that is lower or equal component-wise than . We define as usual the binary vectors : and for every . If , we put . Every can be written as for some . We call such a subset the support of and we denote it . We use the notation and to denote, respectively, the vector of all ’s and the vector of all ’s of any possible dimension.
2 Controlled majority dynamics
We consider a set of players connected by a simple undirected graph (e.g. is such that for any and iff ). The binary set is the set of possible actions for all players. We put and we define the majority game on as the game where each player has utility given by
where is the neighborhood of agent in . In other words, is the number of neighbors of with which is in agreement.
As usual in game theory, given a configuration vector and a player , we indicate with the configuration restricted to all players but and we consequently write .
Best response sets are defined by
Using the notation
to indicate the number of neighbors of an agent playing action in the configuration , Best response sets can be more explicitly described as
denotes the set of Nash equilibria: . This set depends on the topology of the graph , note however that and are always Nash equilibria.
It is well known that this game is potential with a potential given by
This simply says that, for every configuration , action and player , it holds
The (asyncronous) best response dynamics is a discrete time Markov chain (MC) on the configuration space where, at every time , a player is chosen uniformly at random and it modifies its action choosing an element uniformly at random within . Denote with the transition matrix (on ) of the MC and note that, given and , it holds that
From the fact that the potential is not decreasing along the trajectories of , it follows the classical result that, with probability , converges in finite time to the set of Nash equilibria.
The question we pose is: what is the minimal number of agents that if forced to will ensure almost surely that the best response dynamics reach the state .
Given a subset , we indicate with the Markov chain where only the agents in update their action according to the best response rule defined above, while agents in maintain action . This new MC takes values in the subset of configurations
This restricted game remains potential. This new dynamics will converge too to its set of Nash equilibria .
The following definition is the main object of study of this paper.
Definition 1 (Sufficient control set)
is a sufficient control set if
A sufficient control set is minimal if none of its strict subsets is a sufficient control set.
A sufficient control set is optimal if there exists no sufficient control set of strictly smaller cardinality.
Our objective is to find optimal sufficient control sets.
To give a more intuitive idea of what control sets resemble, here are a few illustrative examples.
3 Monotone crusades and valid control sets
We now define the concept of monotone crusade, that will play a crucial role in our theory.
They are particular sequences of states, starting from an initial state, end at and such that at every step, the number of their component with value , without ever decreasing the potential.
Definition 2 (Adapted Monotone Crusade)
Let . A monotone crusade from is a sequence of vectors , for such that
for every there exists such that
Moreover, if given a function , it holds
then the sequence is called a -adapted monotone crusade from .
A few comments on the above definition:
All nodes appearing in (2) are necessarily distinct otherwise the condition for all would be violated. Indeed, we must have that and thus . This allows for a monotone crusade from to be equivalently characterized by the sequence of nodes , the induced order on the nodes or by the sequence of increasing support sets defined by , for having the property that and .
We can also define a decreasing version of the monotone crusade where , and where. for every , . This will be called a decreasing monotone crusade to (-adapted if property 3. holds true).
Definition 5 (Valid control set)
A set is -valid if there exists a -adapted monotonous crusade from .
Main goal of the rest of this section is to show that the class of -valid control sets (we recall that is the potential of the majority game) coincides with the class of sufficient control sets defined in (Definition 1).
The following property is instrumental to our results.
Lemma 6 (monotonicity of Coordination Game)
For all and , the following conditions hold:
if and then ;
if and then .
Proof We only prove the first assertion, the second can be obtained by exchanging the role of and .
If , we have that and there is nothing to prove. If , then the inequality ensure that and we are thus in the previous case.
We now consider the case when both and have value . Note that, if is such that , the variation of the potential when player changes its action from to can be expressed as
where, we recall, and are the number of neighbors of whose action is, respectively, and . As , we have that and . Hence,
This yields the thesis.
Proposition 7 (monotonicity for inclusion)
A superset of a -valid control set is a -valid control set.
Proof Assume that is a -valid control set and let . Let be a -adapted monotone crusade from with associated sequence of points for such that for each . Consider the subsequence of points that are in and put . By construction, we have that and thus, by Lemma 6 and the fact that is a -adapted monotone crusade from , we have that .
Remark 8 (full set).
The full set is always a -valid control set
The following clarifies the connection between valid control sets for the majority game and sufficient control sets introduced in the previous section.
A subset is a sufficient control set iff it is a -valid control set.
Proof We first show that a sufficient control set is -valid. If is a sufficient control set, there exists a sequence of vectors such that and that the best response dynamics follows with positive probability. This is equivalent to saying, using the definition of best response dynamics (see in particular property (4)) , that
for all ;
For every , define
that is the first time when agent change its action to in the sequence . Order now the agents in as in such a way that . Consider the increasing monotone crusade associated with the sequence of points , namely,
and notice that . Since , it follows from Lemma 6 that . This tells us that is a -adapted monotone crusade from and, thus, a -valid control set.
We now show that a -valid control set is sufficient.
We fix any initial condition and we put . is a superset of and, on the basis of Proposition 7, is also a -valid control set. Let be a corresponding -adapted monotone crusade from . By the properties of adapted monotone crusades (properties 2. and 3. in Definition 2) and the characterization (4) of the transition matrix of the best response dynamics , starting from , the MC will follow such a sequence with positive probability. Thus, from any initial condition, will reach with positive probability. A standard result on MC then yields the thesis.
4 Markov chains and backward search algorithms
The characterization of sufficient control sets through the concept of monotone crusades suggest the possibility that such sets can be found starting from the configuration , iteratively replacing ’s with ’s in the attempt to follow backwards a monotone crusade. To this aim we now introduce a family of MC on the binary space , parameterized by that will be the core part of our algorithms.
Transitions of are described as follows:
At every discrete time, a node uniformly at random is activated. If its neighbors with current action () are strictly less than its neighbors with current action (), it stays still. Otherwise, if its action is it changes to with probability , while if its action is , it changes to with probability .
The only non zero non trivial transition probabilities of are the following. Given ,
In the case when , only transitions from to are allowed. In this case, the MC has absorbing points. The relation of these points with sufficient control sets is studied in the next result.
the sets of reachable and absorbing state of the chain .
The following facts hold:
is a sufficient control set iff ;
is a minimal sufficient control set iff ;
(1): By definition, if , there exists a sequence of configuration vectors , for such that and satisfying the properties
for all ;
Then is a -adapted monotone crusade from and this yields that is a -valid control set and thus also a sufficient control set by virtue of Theorem 9. Inverting this argument we prove the other implication.
(2): If is a minimal sufficient control set, we know from (1) that . If, by contradiction, , then, from , the MC could reach, in one step, a different state with . This contradicts minimality.
Theorem 10 allows to reformulate the optimization problem as follows:
where . Optimal sufficient control sets are those for which solves (7).
The MC is naturally related to the minority game whose definition we briefly recall thus: Given an undirected graph , we define the minority game on as the binary game where each player has utility given by
that is simply the number of neighbors with which is in disagreement. We denote by the set of Nash equilibria of the minority game. This game is potential with a potential that is just the opposite than the potential of the majority game:
The following property clarifies the relation of the minority game with our problem.
: Nash equilibria of the minority game are valid control sets.
Proof Let and let , for be any decreasing monotone crusade from to . By construction, for all . Consider the sequence of nodes such that . We have that for every . For all , since is a Nash equilibrium then is in ’s minority best response, thus , or equivalently, . By Lemma 6, it follows that . Therefore is a -adapted monotone crusade from . By virtue of Theorem 10, we have that .
We have the following simple but not obvious consequence.
Corollary 12 (existence)
For any graph, there exist a sufficient control set whose size is less or equal to half the number of nodes.
Proof The minority game, being potential, admits at least one Nash equilibrium . By symmetry is also a Nash equilibrium. Proposition 11 tells us that they are both sufficient control sets and one of the two has the required size property.
In the following figures 4 and 5 we report two examples of optimal sufficient control sets that are not Nash equilibria of the minority game, while figure 6 is a counter example to the assumption that elements are minimal.
The above considerations imply that we can not directly use the dynamics as our algorithm to find optimal sufficient control sets as its absorbing state are not even minimal in general. To overcome this difficulty we will instead use the MC with . As we will see, the presence of the transition in the opposite direction will permit the algorithm to make a full search on the set and not to remain stacked in non optimal configurations. This is guaranteed by the following final result.
Let . The following facts hold:
is ergodic inside the set of states ;
is time-reversible and its unique invariant probability is given by where is the normalization constant;
For , converges in law to a probability measure concentrated on the subset .
Proof (1): We first show that is invariant by . First notice that the transitions that decrease the number of ’s in are also possible in . We thus only have to check the invariance for the transitions that increase the number of ’s in . Fix , suppose that and put . If , it follows from (6) that . Since , there exists a sequence of configuration vectors , for such that and satisfying the properties
for all ;
There must exist such that . Consider now defined by
Notice that Using again Lemma 6, we have that so that .
Ergodicity follows now from the observation that verify the following property: for all transition with non zero probability, also the reversed transition has non zero probability. If , by definition of , they are both reachable by (and thus also by ) starting from . Therefore, with positive probability, under the MC it is possible to move from to and then to .
(2) We need to show that for all , it holds
Since the only non-trivial transitions of are those in (6), we can reduce to check it in the case when, for some , and . In this case we obtain that
This proves the claim.
(3) follows from the expression of .
By virtue of the reformulation (7), we have that, for small and sufficiently large , the MC will be ’most of the time’ in configurations whose support are optimal sufficient control sets. Thsi observation is at the base of a practical algorithm described in the next section.
We have implemented an iterative algorithm based on the MC . For the sake of increasing the speed to convergence, we actually considered a modification of with all trivial self-loop transitions removed. This induces a little modification in the invariant probability, but do not affect the minimal set that is the return of the algorithm. Details will be reported elsewhere. The algorithm keeps track of the best configuration (the smallest ) found so far.
We have applied the algorithm to random realizations of Erdös-Réniy graphs with different number of nodes and probability . For every value of , we ran executions on randomly generated graphs. The algorithm is stopped after iterations, using for epsilon the constant value .
As a point of comparison, we computed a benchmark optimum consisting of the exhaustive optimum for small graphs, and a much longer execution on bigger graphs. Figure 7 shows the average values of the size of the sufficient control sets computed by the algorithm. We compare it with the benchmark optimum and also with the result obtained looking at the very last step of the algorithm. This plot shows a remarkable performance of the algorithm that in linear time gets quite close to the optimum. It also shows that the MC, though fluctuating, as , still remains close to the optimal configurations. The important question of how to tune the parameter for optimize performance has not been addressed here.
Finally notice how optimal sufficient control sets are scaling linearly with respect to the size of the graph. This suggests that Erdös-Réniy graphs are somewhat ’difficult’ to control in this sense, in other terms they show resilience to this type of external actions.
We have formulated the problem of finding, in a network coordination game, the minimum number of players to be controlled in order to drive the system from one Nash equilibrium to another one. To the scope, we have designed a low complexity randomized algorithm and proven its convergence properties. We have finally carried on some numerical simulations corroborating the results.
Many challenging issues naturally pop up from our analysis and simulations. Erdös-Réniy graphs have exhibited optimal sufficient control sets growing linearly in the size of the graph. It would be of interest if this could be proven analytically, as well if this could be extended to other family of graphs, connecting such resilient phenomena to topological properties.
The problem studied in this paper is an instance of a more general problem of studying the effect of control actions in evolutionary game theory. Future research will be in this direction analyzing similar control problems for general games with strategic complementarity as well strategic substitutes.
- Drakopoulos et al. (2014) Drakopoulos, K., Ozdaglar, A., and Tsitsiklis, J.N. (2014). An efficient curing policy for epidemics on graphs. IEEE Transactions on Network Science and Engineering, 1(2), 67–75.
- Drakopoulos et al. (2016) Drakopoulos, K., Ozdaglar, A., and Tsitsiklis, J.N. (2016). When is a network epidemic hard to eliminate? Mathematics of Operations Research, 42(1-14).
- Jackson and Zenou (2015) Jackson, M. and Zenou, Y. (2015). Games on networks. Handbook of Game Theory with Economic Applications.
- Kempe et al. (2003) Kempe, D., Kleinberg, J., and Tardos, E. (2003). Maximizing the spread of influence through a social network. In A.N. York (ed.), KDD ’03 Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, 137–146.
- Morris (2000) Morris, S. (2000). Contagion. The Review of Economic Studies.
- Vassio et al. (2014) Vassio, L., Fagnani, F., Frasca, P., and Ozdaglar, A. (2014). Message passing optimization of harmonic influence centrality. IEEE Transactions on Control of Network Systems, 1(1), 109–120.
- Yildiz et al. (2013) Yildiz, E., Ozdaglar, A., Acemoglu, D., Saberi, A., and Scaglione, A. (2013). Binary opinion dynamics with stubborn agents. ACM Transactions on Economics and Computation, 1(4), 19–30.