Consider an outbreak of a fatal disease. Flu outbreaks happen every year and vary in severity, depending in part on what type of virus is spreading. The influenza or flu pandemic of 1918 was the deadliest in modern history, infecting over 500 million people worldwide and killing 20-50 million victims. Between the years 2014-2016 West Africa experienced the largest outbreak of Ebola in history, with multiple countries affected. A total of 11,310 deaths were recorded in Guinea, Liberia, and Sierra Leone. With such pandemics, global connectedness may trigger a cascade of infections. The outbreak of Ebola which raged in West Africa echoes a scenario where long-range routes of transmission - most prominently, international air routes - can allow the deadliest viral strains to outrun their own extinction, and in the process kill vastly more victims than they would have otherwise. Successful pathogens leave their hosts alive long enough to spread infection. Supplies for vaccinating the vast population may be scarce as opposed to the contagion speed. We search for immunization strategies, attempting to find methods to eliminate epidemic threats by vaccinating parts of a network. Infections may propagate quickly and be discovered only at late stages of propagation.
The analysis of epidemic spreading in networks has produced results of practical importance, but only recently the study of epidemic models under dynamic control has begun. Vaccination policies define rules for identification of individuals that should be made immune to the spreading epidemic. In this paper we study sequential vaccination policies that use full information on the infectious network’s state and topology, under specific budget constraints. The problem of vaccination is central in the study of epidemics due to its practical implications. Viruses, sickness, and opinions can propagate through complex networks, influencing society, with benefits, but also calamitous effects. Viruses spreading over phones and email, online shaming over social networks, human disease via multiple relationships, are merely a few examples of the fundamental necessity of epidemic research.
1.1 Our Contribution and Related Work
The Firefighter problem was first introduced by Hartnell (1995). It is a deterministic, discrete-time model of the spread of a fire on the nodes of a graph. In its original version, a fire breaks out at some node of a given graph. At each time step, nodes can be protected by a firefighter and then the fire spreads to all unprotected neighbors of the nodes on fire (this protection is permanent). The process ends when the fire can no longer spread. At the end, all nodes that are not on fire are considered saved. The objective is at each time step to choose a node that will be protected by a firefighter such that a maximum number of nodes in the graph is saved at the end of the process. The Firefighter problem has received considerable attention (see, e.g., Anshelevich et al. (2012), Cai et al. (2008), Develin & Hartke (2004), Iwaikawa et al. (2011), King & MacGillivray (2010), Ng & Raff (2008)). Develin and Hartke Develin & Hartke (2007) proved a previous conjecture that firefighters per time step are needed to contain a fire outbreak starting at a single node in the d-dimensional grid when . They also found an optimal solution to the case of the 2d grid, with 2 firefighters per time step. Their optimal solution is depicted in Figure 1. The Firefighter problem has been found to be infamously difficult. It is known to be NP-Complete, even when restricted to bipartite graphs (MacGillivray & Wang (2003)) or to trees of maximum degree three (Finbow et al. (2007)), and was proven to be NP-hard to approximate within for any (Anshelevich et al. (2009)). A survey of results on the Firefighter problem can be found in Finbow & MacGillivray (2009).
The control of epidemics has been extensively studied for the past two decades. The dynamic allocation of cure has been studied in Gourdin et al. (2011); Chung et al. (2009); Borgs et al. (2010); Drakopoulos et al. (2014, 2015). The Firefighter problem Hartnell (1995); Develin & Hartke (2007); Esperet et al. (2013); Finbow et al. (2007); Anshelevich et al. (2009); MacGillivray & Wang (2003); King & MacGillivray (2010); Cai et al. (2010) and this paper do not model healing processes of individuals, making the task of finding an optimal vaccination strategy dependent only on the spreading infection process and the underlying graph. Not including a “natural healing” process while only allowing vaccination of non-infected individuals, differentiates the Firefighter problem and our problem from these stochastic epidemic models. More related to our work, are the studies of vaccine allocation in Cohen et al. (2003); Preciado et al. (2013); Peng et al. (2013); Ruan et al. (2012); Miller & Hyman (2007); Peng et al. (2010); Bodine-Baron et al. (2011). Victor M. Preciado et al. Preciado et al. (2013), propose an optimization strategy for optimal vaccine allocation. Their model assumes slightly modifiable infection rates and a cost function based on a mean field approach. Information-driven vaccination is studied in Ruan et al. (2012), showing the spread of the information will promote people to take preventive measures and consequently suppress the epidemic spreading. Miller JC and Hyman JM propose in Miller & Hyman (2007) to only vaccinate those nodes with the most unvaccinated contacts. An acquaintance immunization policy was proposed in Cohen et al. (2003), where it was shown that such a policy is efficient in networks with broad-degree distribution. A different approach Bodine-Baron et al. (2011), considers minimizing the social cost of an epidemic. The above studies’ analysis is based on epidemic thresholds and mean-field approximations of the evolution process. In contrast to these, this paper can be viewed as a stochastic extension to the Firefighter problem, which studies the transient, short-term behaviour of a spreading infection.
Another variant of the Firefighter problem proposed by Anshelevich et al. (2012), assumes the vaccination is also a process that spreads through the network. In the spreading vaccination model, vaccinated nodes propagate through the network as well. If a node is adjacent to a vaccinated node, then itself becomes vaccinated during the next time step (unless it is already infected or vaccinated). This type of model may depict conflicting ideas co-existing in a social network or the fact that vaccines can be infectious as well, since they are often an attenuated version of the actual disease. Anshelevich el al. Anshelevich et al. (2012) show that the “spreading vaccination” problem can be written as a maximization problem of a monotone submodular function over a matroid. These optimization problems are known to have a greedy algorithm which achieves a constant factor approximation Krause & Golovin (2014). The reader may refer to Anshelevich et al. (2012) for an exhaustive account of the problem. This paper focuses on the original Firefighter problem, without spreading vaccination.
Previous research on immunization policies modeled the problem either deterministically or approximately, considering long-term effects alone. Defending a network from an attack of a virus must take into account its transient behavior, fast propagation, as well as the natural budget constraints of defending the network. It is thus vital to find on-line policies which acknowledge short-term effects and the defender’s limited protection capabilities. This differentiates our paper from previous research, and takes a leap towards exact analysis, under a transient probabilistic framework.
Our contributions are as follows.
We extend the original Firefighter problem to a stochastic framework and define optimality criteria for (a) number of infected individuals and (b) budget needed for containment.
We propose several approaches for analyzing immunization and containment of “fast-moving” (i.e., not infinitesimally slow) epidemics in networks. This is in contrast to the common assumption of small infection rates. We then develop upper and lower bounds for these.
We construct an algorithm for state dependent budget (i.e., budget allocation which changes according to the state of infected and vaccinated individuals at time ).
The paper is organized as follows.
In Section 4 we propose a simple greedy policy, which only chooses to vaccinate nodes neighboring the infection. An acquaintance immunization policy was proposed in Cohen et al. (2003), where it was shown that such a policy is efficient in networks with broad-degree distribution. This greedy policy can be efficiently calculated on any graph, and is thus of high interest. We show such a policy is optimal when either the network topology is a regular tree, or the graph is finite and the infection spread is sufficiently slow.
In Section 5 we develop mathematical machinery to characterize the topology-dependent speed. We define growth rates, maximal minimal growth rates, as well as expected growth rates. We show in Theorem 5.1 how these quantities can be used to bound the expected limiting cardinality of an infection under any vaccination policy.
We apply the results of Theorem 6.1 to specific network topologies, including regular trees, -dimensional grids, and Erdős-Rényi graphs to obtain explicit bounds on the budget needed for containment. We summarize these results in Table LABEL:table:summary.
In Section 6.2 we propose an algorithm for state-dependent budget allocation, based on estimated local growth rate behavior of the network. We show this strategy achieves better containment on two real world networks, when compared to a constant budget strategy which consumes an equal global budget.
This section defines the model of our problem and the criteria we wish to optimize. A natural way to extend the Firefighter problem to a probabilistic setting is to model it as an MDP (Bertsekas et al. (1995)). The NP-Completeness of the Firefighter problem impedes much for its research. Modeling the problem as probablistic, as well as defining different objectives to the problem, may allow us to obtain stronger results. We will distinguish between two different criteria. The first, and most natural criterion, considers minimizing the number of infected nodes at the end of the infection process. This criterion, which is the same as the original Firefighter problem objective, is hard to optimize generally. We consider an alternative criterion, which is ultimately easier to evaluate. The containment criterion asks the following question: What is the minimal budget needed to ensure an infection won’t reach a certain size? This question can be answered immediately once an optimal policy is known, whereas knowing the latter does not give us the former. Strictly speaking, this question may be more feasible to answer, and is thus the main focus of our work. In regard to the optimality criterion, we find upper and lower bounds on the optimal loss.
2.1 Stochastic Firefighter Model as an MDP
We model the problem as a discrete SIR epidemic model, defined by parameters (graph network topology),
(infection probability / speed), and(vaccination budget per stage). We consider a network, represented by an undirected graph , where denotes the set of nodes and denotes the set of edges. Two nodes are said to be neighbors if . We use the notation to denote neighboring nodes. We use to denote the number of nodes in , and do not restrict ourselves to finite graphs.
Assume a spreading infection on
to be a discrete time contact process, modeled by a discrete time Markov chain. An infected node infects each of its healthy neighbors with probability . A state of the chain, , is defined as the pair , where and denote the set of infected and vaccinated nodes at time , respectively. At all times, (i.e., a node can either be healthy, infected, or vaccinated).
The process is initialized at some given state with transitions occurring independently according to the following dynamics
If a node is infected, it remains infected forever.
If a node is vaccinated, it remains vaccinated forever.
If a node is healthy at time , then node stays healthy at time with probability , where
This in turn means node moves to the infected state at time with probability
Informally, conditions 1 and 2 mean that nodes remain in their infected or vaccinated states at all times. Condition 3 means that at each iteration nodes infect each of their healthy neighbors independently with probability .
Our MDP is defined by a state space given by tuples that are subsets of , action space which is a subset of of maximum cardinality , a transition probability matrix as defined by the infection dynamics, and a cost function with a penalty of for each infected node. A stationary Markov control policy determines at each state what set of healthy nodes to vaccinate. Once a set of nodes is chosen to be vaccinated they are added to . We impose a budget constraint of the form
to all . More specifically, our action space is a subset of the healthy nodes in state with cardinality at most .
To remove ambiguity in regard to the order of transitions, we assume that at each time step , policy first chooses the set of nodes to vaccinate , after which remaining healthy nodes are infected by their neighbors according to the contagion dynamics. We denote the set of all stationary policies by .
Since any reasonable policy uses all of its budget at each iteration, we restrict ourselves to a subset of , which we denote by , such that all vaccinate exactly nodes at each iteration. That is,
2.2 Optimality Objective
We define an optimal policy as one which minimizes the expected number of infected nodes once there are no more nodes to vaccinate and infect. Formally, for an initial state , we define the expected loss of a policy by
where denotes the expected value induced by policy . For finite graphs, as
, an absorbing state will be reached almost surely. Then, the loss function can also be written as
is a random variable that depicts the time in which an absorbing state is reached under policy(i.e., when there are no more healthy neighbors to infect). When we write in place of . We define the optimal loss by
When it exists, we define an optimal policy by
2.3 Containment Objective
We also investigate whether an infection can be contained (in expectation). We consider two different questions. For inifinite graphs, we ask: can an infection ever be contained under a finite budget ? (i.e., will it grow forever or not?). On finite graphs, we consider containment with thresholds. That is, given a maximal cardinality of infected nodes (i.e., a worst-case “acceptable” loss), is a budget sufficient to ensure an infection does not grow to be larger than that cardinality? The second question applies to infinite graphs as well. Specifically, given a threshold , we wish to find a minimal budget such that when , or when . Formally we have:
Let . Let be a graph on nodes, and be an initial state. When , we say that contains in if . When , we say that contains if .
We say that is a weak upper bound if for any , contains in . We say that is a weak lower bound if for any , does not contain in . We say is a tight bound if is both a weak upper and lower bound. We define equivalently for the case of and denote the containment bound by .
For discrete-time -valued process, we denote by We use the notation for any to denote the positive part of . We denote the set of edges in a graph by and use the notation to denote . We denote by the maximal node degree in a given graph. The number of neighbors of a set of nodes is defined to be the number of connected nodes to that set, and is denoted by for some . In a similar way, for a state , its neighborhood is defined as the set of nodes
Another important measure for sets is their cuts. A cut of a set of nodes in a graph is defined to be the number of edges exiting the set. For any two sets and a node , the cut from to is defined by
The cut from to is then defined by
We can then define the cut of as the cut from to , that is,
Equivalently, for a state and node , the cut from to is defined by
and the cut of by
4 First-Order Policies
Since finding optimal structural policies on general graphs is NP-Hard, we consider ones that are both tractable and intuitive. We propose a simple policy and prove its optimality on trees.
A policy is called a first-order policy if at each time it only vaccinates a subset of nodes in the immediate neighborhood of . That is, .
Under first-order policies, the number of potential states we must consider decreases, simplifying the problem of finding optimal policies. In many cases, though, such policies are sub-optimal. We provide such an example in Figure 2. Consider the graph in Figure 2. Next assume, for clarity of the illustration, that infections proceed deterministically (i.e, ). In such a case, it is straightforward that an optimal policy would vaccinate nodes at level 3 at time , then nodes at level 3 at time . Any other policy would ensure the infection reaches level 4, thus obtaining a loss that is of order , which may be very large. Characterizing settings where first-order policies fail or succeed is a key contribution of this work.
It is interesting to distinguish, when it is optimal to only consider such first-order policies.
The optimal first-order policy is defined by
Two natural first-order policies are the CUT policy and the random policy.
The CUT policy is a first-order policy which minimizes the immediate cut, that is
The random policy is a first-order policy defined by uniformly choosing any nodes in .
The CUT and random first-order policies are easy and efficient to calculate. Although they are only an approximation of an optimal first-order policy, their simplicity brings forth the natural question: when is using a first-order policy a good approximation? In the next subsections we will show two instances in which an optimal first-order policy is optimal, and in further sections use first-order policies to achieve bounds for containment, the optimal loss, and the first-order loss. We will ultimately use these containment bounds in Section 6.2 together with a first-order policy to achieve a state dependent budget allocation strategy.
4.1 Optimality of Slow Propagation
The following theorem is a cornerstone for the rest of this paper.
Let be a finite graph on nodes, and let be an initial infection on . Then there exists a positive value such that is optimal, for every .
That is, on finite graphs, there exists a small enough such that a first-order policy always achieves the smallest loss. The proof reveals, moreover, that a first-order policy may be a good approximation of an optimal policy, as long as is not too big. Thus, the result of Theorem 4.1 underscores the importance of studying fast growing infections as well. Equivalently, one may examine the ratio . We would expect problems with a small ratio to be much more simple (i.e., small and large ). A main focus in this paper is studying containment objective which do not confine us to the “small ” or “large ” regimes.
Let be a state. Assume by contradiction there exists a stationary policy such that . Let be a first-order policy as defined in equation 2. Without loss of generality, suppose , so that . Also, for clarity of the proof, and without loss of generality, assume . Throughout this proof we will use to denote the minimal distance between a set and a node .
Since is a finite graph, an absorbing state is reached with probability . We thus use the notation to denote , where , as defined in Section 2.2, is the random variable depicting the time an absorbing state is reached, that is the time under which . To prove optimality of the first-order policy, we show there exists a small enough , such that vaccinating in ensures a smaller loss. For a budget it easy to see that is upper bounded by a.s. For the remainder of the proof we will use the notation to denote that upper bound. Let with , and let be the number of paths in leading from to of length at most . We define the event by
Since , it must be that . Then,
Next we define the event
and note that can be written in terms of events from Equation (4), as
For brevity, we will use the notation . Then we have
where in the first inequality we used the union bound, and in the second we used the inequality in Equation (5). Since does not vaccinate at least one node in , the immediate loss of compared to can be bounded by
Furthermore, given the event ,
This is true because given the event all nodes that are of distance greater than will never be infected, thus any policy which does not vaccinate at time in will do at least worse than a first-order policy.
Next note that
Furthermore it trivially holds that
Finally let , and take . Then
where here the first inequality uses Equations (9) and (10). In the second inequality we have used the fact that as well as the inequality in equation 6. Finally, the last strict inequality uses the fact that .
We have reached a contradiction to the optimality of , thereby completing the proof. ∎
4.2 Optimality on Trees
We show next that regardless of the transmission probability , first-order policies are optimal on trees.
Let be a tree, and suppose an infection initiates at its root. Then is optimal.
Let denote the subtree of a node in the tree, and note that
Assume by contradiction that the first-order policy, , is sub-optimal. Let be an optimal policy that is not a first-order policy. Then there is a state from which vaccinates a node outside the neighborhood of . Denote this node by . Then, there exists a node such that is in a shortest path between and , which is also a unique path, since is a tree. Also, we have . Let be the policy which vaccinates instead of at time . Then by these facts, and using Equation (11),
We can continue this procedure until . Therefore, is a first-order policy, and achieves a minimal loss, which contradicts our assumption that is sub-optimal. ∎
4.3 First-Order Policy for Trees
Theorem 4.2 shows us that for trees, a first-order policy is always optimal. Finding this optimal first-order policy, however, can still be NP-Hard (see Finbow et al. (2007)). We therefore look for instances of trees where an explicit structural policy can be established. For this reason, we consider the case of -regular trees.
Let be a regular tree of degree . We denote by the level (from the root) of a node , where we consider the root to be of level 1, that is, . We also denote the number of levels by , where we allow . We assume an initial infection spreads in such that is a connected set. We define the set as
Note that once it must be that .
In order to obtain exact analytical results, we disregard edge effects by assuming is deep enough such that the infection never reaches its leaves. That is, we implicitly assume a limit of . Algorithm 1 outlines an optimal structural policy for regular trees, which is indeed a first-order policy since it only considers nodes in . We prove this policy is indeed optimal in the following Theorem.
Let be a regular tree, and let be a connected set. Then the tree policy outlined in Algorithm 1 is optimal.
Recall that throughout all of our derivation we assume is a connected set, so that must also be a connected set for all .
Denote by a subtree of with its root at level of . Also denote by the optimal loss for an infection spreading on the graph with an infection initiated at its root. It follows that
Let with . Using the fact that together with equation 12 yields .
Case 2: and
We create a new tree as follows. Take all nodes in and merge them together to one node such that all neighbors of remain neighbors of . is an asymmetric tree with as its root. One of the children of is . Since it has exactly healthy children in . This means we can use equation 12 of Case 1 for (as the root is contained in ). Hence, since is of lower level than all other children of , it must be vaccinated first.
Case 3: and
In this case, we again construct as in case 2. Once is created, will have healthy children, while all other children of will have children. This comes into conflict with Equation (12) which deals with a case of all nodes having the same degree.
Since we only consider the case of , an optimal policy must choose to vaccinate nodes with higher degree, that is any nodes in the neighborhood of excluding , leaving as a last priority for vaccination. A visualization of the process of building for this case is depicted in Figure 4.
Theorems 4.2 and 4.3 give us optimal policies on trees, but do not tell us the actual expected loss, nor do they give us estimates on the needed budget for containment. In order to obtain bounds on these quantities, and also to study general (non-tree) topologies, we develop the concept of growth rates in the next section.
5 Growth Rate
This section develops the required machinery for developing upper and lower bounds on the optimal loss. These bounds enable us to answer questions regarding containment objective. We define the notion of the growth rate. Informally, the growth rate of an infection at time is the expected cardinality growth of between time and . That is, if we denote by the growth rate at time , then
The growth rate thus changes at each time step . Intuitively, a good policy would (1) maintain a small growth rate throughout the propagation of the infection and (2) bring the growth rate to zero as quickly as possible. We define the notion of the growth rate formally below.
The growth rate of a set to a set for infection speed is defined by
The growth rate of a set is defined using equation (14) as . Equivalently, we define the growth rate of a state to be .
In order to get a better understanding of the growth rate, we note the following simple lemma, which states that the number of neighbors and cut of a state behave as lower and upper bounds of the growth rate of that state, respectively.
Let be a state. Then .
For any and ,
On the other hand, for any ,
Also, if then . Then for any ,
The growth rate of a state , , takes into account the vaccinated set of nodes . It can be useful to write it using the growth rate for sets in the following way:
5.1 Maximal and Minimal Growth Rates over Upward Crusades
Growth rates give us a tool for measuring the expected growth of an infection at a time , but do not take into account the growth rate in future states. In order to build better structural policies, we consider the full propagation of the epidemic using the notion of maximal and minimal growth rates. These are merely parameters that tell us how large or how small the growth rate can be under a specific policy in a given time window. To define these, we follow Drakopoulos et al. (2014), and define upward crusades as sequences of possible states under a given policy. We then define the maximal and minimal growth rates as the worst and best cases of growth rates over all possible sequences.
An upward crusade is a sequence of infection states which follow all possible realizations of infections starting at some initial state under some policy . Upward crusades are important as they enable us to define maximal and minimal growth rates, which then enable us to obtain bounds on the optimal loss.
For a state and policy , an upward crusade of length is a sequence of pairs (states), , with the following properties:
We denote the final state of the sequence by . We also denote the set of all upward crusades of length initiating at a state by . Finally, we denote the set of all possible upward crusades of length for the set of policies by , that is
It is sometimes helpful to consider upward crusades that do not vaccinate any nodes. These upward crusades depict worst case instances of infection states in the graph under arbitrary policies. When we wish to consider such upward crusades, we assume , and denote them by , where here we omit (since ), and use to denote the initial set of infected nodes. Figure 4 depicts examples of upward crusades on a simple graph.
Remark 1 (Inclusion of upward crusades).
Note that for any state and , with there exists such that .
The following lemma is an important monotonicity property of upward crusades.
Lemma 5.2 (Monotonicity in ).
Suppose . Let . Then there exists
such that and .
Let . We build an upward crusade as follows. The first sets of the sequence are the same as . In the final sets we follow the upward crusade satisfying . This in turn results in . ∎
Maximal and minimal growth rates let us bound the expected loss of a policy. Using upward crusades, the maximal growth rate looks steps into the future, until the point when an infection of cardinality is reached under some policy . It then returns the worst case growth rate over all corresponding upward crusades. This enables us to bound the loss of the policy, as will be shown in Theorem 5.1. A similar idea follows for the minimal growth rate. We define these formally in the following definition.
Definition 5.3 (Maximal and Minimal Growth Rates).
Given a policy , a state , and the set of upward crusades of length , , the maximal and minimal growth rates are functions , defined by
where , and recall that denotes the final state in an upward crusade . In other words, the maximum / minimum growth rates maximize the growth rate over all states that end upward crusades in and have infection cardinality .
Definition 5.4 (Expected Growth Rate).
Given a policy , a state , and the set of upward crusades of length , , the expected growth rate is a function , defined by
where is the random variable of the final state in upward crusades of , such that the integral is taken over final states with cardinality .
It is helpful to consider maximal, minimal, and expected growth rates of an empty policy which does not vaccinate any nodes. We will denote these using upward crusades in place of to emphasize the fact the maximum/minimum/expectation is taken over crusades which do not vaccinate any nodes (i.e., ). This notation is useful for finding bounds for specific topologies (see Section 6).
(b) Plots of (in blue), (in red), and (in green) as functions of , with . Estimates were made using Monte Carlo simulations (see Section 6.2 for details).
5.2 Upper and Lower Bounds
Calculating the loss of a policy as well as the optimal loss can be done using Monte Carlo simulations. Finding analytical expressions, though, is in most cases not possible. To overcome this problem, we look for a method of bounding the loss of a policy. Theorem 5.1 explicitly calculates upper and lower bounds for the loss. These bounds allow us to obtain specific conditions for containment, as we explain in Section 6.
Let be a graph on nodes, and let be an initial state. We wish to bound from above the loss of a policy . We define a process as the expected cardinality of the infection at time given the initial state and the policy, that is
is directly linked to the loss of a policy . The loss of a policy can be written in terms of as . Equation (13) gives an interpretation of the growth rate which we can leverage to calculate and consequently obtain . This calculation is in general intractable. Thus, we turn to maximal, minimal, and expected growth rates to bound from below or above (Section 6.2 also gives a method of approximating these bounds). Our main approach includes defining recursion relations which lower/upper bound the process . We define the functions
We use linear interpolation on the cardinality parameter, and define the following recursion relations ():
where is a given parameter.
We denote by the maximal in which the conditions of recursion (19) are satisfied, namely, . The following theorem gives general bounds on the process .
Theorem 5.1 (Bounds).
Let , be defined above.
Suppose is concave and monotone non-decreasing in , . Then for any .
Suppose is convex and monotone non-decreasing in , . Then for any .
Suppose is concave (convex) and monotone non-decreasing in ,
Then () for any .
We prove by induction on .
Assume that for some . By definition of , the maximal growth rate function, and for any state with ,
Next we bound by
where in inequality (a) we used concavity of in for , and in inequality (b) monotonicity of in for as well as the induction step.
The proof follows the same steps as in 1.
By definition of , the expected growth rate function, and for any state with ,
If is convex then assume that for some , then using the induction step and monotonicity of the result is obtained as in 1. If is concave then assume that for some , then u