## I Introduction

The study of epidemics on networks is an active research topic (e.g., see [1, 2, 3] and the references therein). Real-life examples include the propagation of burst errors in a wireless communication channel [4], of a biological disease through a population [5], of malware in computer or smartphone systems [6], and the dissemination of rumors [7] and competing opinions [8] in social networks.

In this paper, we examine two player zero-sum games over Polya contagion networks as introduced in [9, 10]. The setup for this model is an adaptation of the classical Polya contagion process [11, 12, 13] to allow for spatial interactions between neighbouring nodes. This model is similar to the well known class of Susceptible-Infected-Susceptible (SIS) epidemics models [1]. Indeed in [10], it was empirically shown that the network Polya contagion model can mimic the behaviour of the SIS process. Furthermore, SIS models have been studied in a game-theoretic context in [14, 15] and references therein. We herein establish a similar problem for the network Polya contagion model.

First, we define a two-player game on the average infection rate for a connected network. This setup is an extension of the contagion curing problem that was studied extensively in [16]. In order to simplify the problem of finding optimal control policies, we consider a game on the proxy measure of the expected network exposure. We establish that the expected network exposure is convex as a function of the curing parameters and concave as a function of the infection parameters. We prove that under budget constraints, there exists a Nash equilibrium for the game on the expected network exposure, which can be determined using gradient descent algorithms. Finally, we run a series of simulations on small test networks, to provide empirical evidence that a Nash equilibrium exists for the infection-curing game on the average infection rate, and to support the claim that such an equilibrium policy can be closely approximated using the equilibrium policy for the expected network exposure.

The rest of the paper is organized as follows. Section II familiarizes the reader with important preliminary material, including a brief description of the classical Polya contagion process as well as the network Polya contagion process. Section III defines the two-player games that are studied in this paper, and proves important convexity/concavity results as well as the existence of a Nash equilibrium for the expected network exposure game. Section IV presents illustrative simulation results. Finally, Section V concludes the paper and highlights a few areas for potential future research.

## Ii Preliminaries

Let

be a probability space. Consider the stochastic process

, whereis a random vector on

. We represent by , the natural filtration on the process . We write the sequence as , and more generally we use the notation , where . Further clarification of standard probability concepts can be found in various texts such as [17, 18].A Nash equilibrium is a solution for a non-cooperative multiplayer game from which no player may improve its payoff by unilaterally changing its strategy [19, 20]. In this paper, we consider games with deterministic (pure) strategies. In the context of a two player zero-sum game, denote by and the set of allowable strategies for player one and two, respectively. More precisely, consider the case where player one receives a payoff , with and strategies of player one and two, respectively. Then, is a Nash equilibrium if , :

### Ii-a Classical Polya Contagion Process

We recall the classical Polya contagion process [11, 12, 13]. An urn contains red balls and black balls. We use to denote the total number of balls in the urn. At each time, , a ball is drawn from the urn and returned with balls of the same colour. From an urn sampling point of view the parameter

is integer-valued; however, we allow them to be real-valued for mathematical convenience (e.g., when determining optimal policies). The random variable

is used to indicate the colour of the ball on the th draw, where if the drawn ball was red and if the drawn ball was black. We can then use to denote the proportion of red balls in the urn after the th draw, wherewhere is the initial proportion of red balls in the urn and is a correlation parameter. The conditional probability of drawing a red ball at time , given the past history of draws , is given by

### Ii-B Network Polya Contagion Process

We now review the network Polya contagion process introduced in [9, 10]. Let be a network, where is the set of nodes and is the set of edges. Throughout, is taken to be undirected, and is assumed to be connected. The set of neighbours to node is given by . We then define . Each node is assigned an urn with red balls and black balls, with the total number of balls at node being denoted by .

We next introduce the concept of super urns for node , which contains the red and black balls at node as well as the red and black balls of each of its neighbouring nodes. This concept is illustrated in Figure 1. For each node , we use and to denote the number of red and black balls respectively in node ’s super urn. The total number of balls in the th super urn is given by . Similar to the classical Polya process, a draw is conducted for each node at each time, and then a number of balls of the same colour is added to that node’s individual urn. In this case, however, the draw is conducted on the super urn of each node. As well, we may allow for the number of added balls to vary based on which node the draw was for, the colour of the draw, and the time at which the draw occurred. Thus, for node at time , if a red ball is drawn we add red balls to node ’s individual urn and if a black ball is drawn we add black balls to node ’s individual urn. We use an indicator, to denote the colour of the th draw for node , where if the draw was red and if the draw was black. We refer to as the curing parameters and as the infection parameters for node .

For this process, we can represent the proportion of red balls in node after the th draw by , where

Here

(1) |

represents the total number of balls in node ’s urn after the th draw. We represent the proportion of red balls in node ’s super urn after the th draw as , where

(2) |

We take , which is just the initial super urn proportion of red balls for node . We define

which we refer to as the network exposure.

The conditional probability of drawing a red ball for node at time , given the past history of draws for all the nodes

is given by

Finally, we refer to

as the average infection rate.

## Iii Game-Theoretic Setup

Previously, [16] has looked at the problem of minimizing the limiting average infection rate subject to a budget on the curing parameters at each time step. In this setup, is fixed for each time step and all other values are assumed to be given. We now consider a two player game, where player one looks to minimize the average infection rate, while player two looks to maximize this same value. Player one controls the distribution of curing parameters in accordance with a budget, , while player two controls the distribution of the infection parameters in accordance with a budget, . In both cases the budget is fixed for all time steps, and for any given time step the allocation of the budget is performed prior to the draw. Thus, if either player allocates resources to a node for which the opposite colour ball is drawn, those resources will go to waste.

Formally, player one’s objective is as follows:

(3) |

assuming the minimum exists, while player two’s objective takes the form:

(4) |

assuming the maximum exists.

For a general network, finding an optimal control policy for either player can be complicated, and the behaviour of is difficult to study. As discussed in [16], there is a strong correlation between the behaviour of and , and thus we can simplify our problem by first considering a game on the expected network exposure for a given time step, rather than the limiting average infection rate.

We consider the expected network exposure as a two player zero-sum game, where player one looks to minimize the expected network exposure over the curing parameters and player two looks to maximize the same value over the infection parameters . We begin with an important result about the expected network exposure:

###### Proposition III.1

(Convexity-Concavity of Network Exposure): Let be a general network, and consider the Polya network contagion model on , with arbitrary initial conditions. Then, the expected network exposure is convex with respect to the curing parameters and concave with respect to the infection parameters , for all .

Alternatively, let represent the adjacency matrix (including self connections) for our given network , where iff , and , otherwise. Then construct an square matrix, , where . Letting represent the row of the matrix , we get , where

Likewise, we can construct an square matrix, , where . Letting represent the row of the matrix , we get the following:

Allowing us to alternatively write

noting of course that the matrices and are functions of .

Taking the expectation of given the history of the contagion process up to time we get

(5) |

We can note that is independent of our choice of , and for any fixed realization , is convex in over . Hence, it follows that is convex in over , and thus so too is . Concavity in follows from a symmetry argument wherein we note

is convex in , thus showing is concave in . The rest of the proof follows as in the convexity argument for .

The natural symmetry of this model makes this result rather intuitive, and allows for us to establish a nice setup for the game theoretic problem. We have proven that is convex in and concave in over . In order to get a better understanding of this function, we consider the partial derivatives with respect to and . We note that

and furthermore

Since and can only take values in , we get that and . Using this fact in conjunction with equation (III) we determine that over the space , has no saddle point. This lack of saddle point is demonstrated in Figure 2, which depicts the shape of for a single node network. However, given our fixed allocation budgets and , we restrict ourselves to considering sets of the form .

Returning to our game over the expected network exposure, we can then note that for any given , the sets and are convex and compact. This gives rise to the following result:

###### Theorem III.2

(Nash Equilibrium for Network Exposure): Let be a general network equipped with the Polya network contagion model, with arbitrary initial conditions. For a given time , consider a two-player zero-sum game where player one tries to minimize over the parameters and player two tries to maximize over the parameters . Then, if we take our set of allowable policies to be of the form , the resulting game admits a Nash equilibrium. Moreover, the equilibrium policy will satisfy and .

Since the function is convex-concave and over a compact set, the existence follows from the classical minimax theorem, see [21]. By the definition of , and since the function has no saddle point in the interior of its domain, the optimal policy will utilize the full budget.

## Iv Simulations

We have proven that there exists a Nash equilibrium when we consider the two player infection-curing game for the expected network exposure . This however, does not guarantee that the same holds true when considering a similar setup for the average infection rate as in (3) and (4

). We need to check first of all, that we can find policies for each player that mimic the behaviour of a hypothetical equilibrium policy. As well, we want to see if the optimal policies for the network exposure provide a good estimation for our hypothetical equilibrium policy.

We consider three basic networks, as depicted in Figures 3(a), 3(b), and 3(c). Each network is assigned a uniform initial distribution for both red and black balls. A fixed budget of

is given for both players at each time step. At any given time, the optimal policy for the infection-curing game over the expected network exposure is calculated for both the curing and infection parameters. The first set of simulations uses this optimal policy for both players. In the second set of simulations only the curing parameters are assigned using this optimal policy, while the infection parameters are distributed uniformly. In the final set of simulations, the infection parameteres are assigned using the optimal policy, while the curing parameters are uniformly distributed. At each time

, the draw values are averaged over 1000 trials, and then averaged over all nodes to obtain the empirical average infection rate . The plots in Figure 3 display the resulting output for each network plotted against time.For all three networks, we observe that case 3 generally leads to the highest average infection rate and case 2 results in the lowest average infection rate. While we do not know what the optimal policy for either player would be in the case of , we can see that the behaviour observed suggests that a Nash equilibrium exists. This is less obvious when we consider the network in Figure 3(c). For this network, it makes sense that a hypothetically optimal policy would be close to uniform, and so we observe less distinct separation between each of the cases in Figure 3(f) as compared to Figure 3(d) and Figure 3(e). The results in Figure 3(f) do not, however, preclude the possibility that there exists an equilibrium policy for both players.

In general, the results support the notion that a Nash equilibrium exists for the game over the average network infection, , and that using the solutions obtained from the gradient descent over the expected network exposure may provide a good estimation as to what such an equilibrium policy may look like. However, the desire to guarantee such claims necessitates further follow up, as these simulations provide only a narrow view.

## V Conclusions and Future Directions

In this paper, we explored two player zero-sum games over Polya contagion networks, verifying analytically the existence of a Nash equilibrium for an infection-curing game on the expected network exposure, and providing empirical evidence to support the existence of an equilbrium point for the infection-curing game on the average network infection. Future work may further investigate the existence of such an equilibrium point. Considering variations of this problem may also be of interest; for example, it may be useful to explore the infection-curing game under the restriction that only a single node be provided resources, or to consider the case where the underlying ball distribution is unknown to the participating players. Such problems could be considered for both the expected network exposure and average infection rates. The infection-curing game on the average infection rate can also be further broken down into exploring both finite and infinite horizon cases.

## References

- [1] D. Easley and J. Kleinberg, Networks, Crowds and Markets: Reasoning about a Highly Connected World. Cambridge Univ. Press, 2010.
- [2] P. V. Mieghem, J. Omic, and R. Kooij, “Virus spread in networks,” IEEE/ACM Trans. Netw., vol. 17, no. 1, pp. 1–14, 2009.
- [3] C. Nowzari, V. M. Preciado, and G. J. Pappas, “Analysis and control of epidemics: A survey of spreading processes on complex networks,” IEEE Control Systems Magazine, vol. 36, pp. 26–46, 2016.
- [4] F. Alajaji and T. Fuja, “A communication channel modeled on contagion,” IEEE Trans. Inf. Theory, vol. 40, no. 6, pp. 2035–2041, 1994.
- [5] L. Kim, M. Abramson, K. Drakopoulos, S. Kolitz, and A. Ozdaglar, “Estimating social metwork structure and propagation dynamics for an infectious disease,” in Proc. Int. Conf. Social Computing, Behavioral-Cultural Modeling, and Prediction, pp. 85–93, Springer, 2014.
- [6] M. Garetto, W. Gong, and D. Towsley, “Modeling malware spreading dynamics,” in Proc. Conf. Comp. Comm., vol. 3, pp. 1869–1879, 2003.
- [7] E. M. Rogers, Diffusion of Innovations. 5 ed., 2003.
- [8] E. Adar and L. A. Adamic, “Tracking information epidemics in blogspace,” in Proc. IEEE/WIC/ACM Int. Conf. Web Intelligence, pp. 207–214, 2005.
- [9] M. Hayhoe, F. Alajaji, and B. Gharesifard, “A Polya urn-based model for epidemics on networks,” Proc. 2017 American Cont. Conf., 2017.
- [10] M. Hayhoe, F. Alajaji, and B. Gharesifard, “A Polya contagion model for networks,” IEEE Trans. Cont. Netw. Sys., vol. 5, pp. 1998–2010, 2018.
- [11] F. Eggenberger and G. Polya, “Über die statistik verketteter vorgänge,” Z. Angew. Math. Mech., vol. 3, no. 4, pp. 279–289, 1923.
- [12] G. Polya and F. Eggenberger, “Sur l’interprétation de certaines courbes de fréquences,” Comptes Rendus C. R., vol. 187, pp. 870–872, 1928.
- [13] G. Polya, “Sur quelques points de la théorie des probabilités,” Annales de l’institut Henri Poincaré, vol. 1, no. 2, pp. 117–161, 1930.
- [14] A. R. Hota and S. Sundaram, “Game-theoretic protection against networked sis epidemics by human decision-makers,” IFAC-PapersOnLine, vol. 51, 03 2017.
- [15] P. E. Paré, J. Liu, C. L. Beck, A. Nedić, and T. Başar, “Multi-competitive viruses over static and time-varying networks,” in 2017 American Control Conference (ACC), pp. 1685–1690, May 2017.
- [16] M. Hayhoe, F. Alajaji, and B. Gharesifard, “Curing with the network Polya contagion model,” Proc. 2017 American Cont. Conf., 2018.
- [17] R. Ash and C. Doléans-Dade, Probability and Measure Theory. 2000.
- [18] G. R. Grimmett and D. R. Stirzaker, Probability and Random Processes. Oxford Univ. Press, 3 ed., 2001.
- [19] J. Nash, “Equilibrium points in -person games,” in Proceedings of the National Academy of Sciences USA, vol. 36, pp. 48–49, 1950.
- [20] J. Nash, “Non-cooperative games,” The Annals of Mathematical Statistics, vol. 54, pp. 286–295, 1951.
- [21] D. P. Bertsekas, A. Nedić, and A. E. Ozdaglar, Convex Analysis and Optimization. Belmont, MA: Athena Scientific, 1st ed., 2003.
- [22] D. P. Bertsekas, Nonlinear Programming. Belmont, MA: Athena Scientific, 2nd ed., 1999.