In this paper we consider a general class of stochastic games in which every player has an individual state that impacts payoffs. Historically, Markov perfect equilibrium (MPE) has been a standard solution concept for this type of stochastic games (Maskin and Tirole, 2001). However, in realistically sized applications MPE suffers from two drawbacks. First, because in MPE players keep track of the state of every competitor, the state space grows too quickly as the number of players grows, making the analysis and computation of MPE infeasible in many applications of practical interest. Second, as the number of players increases, it becomes difficult to believe that players can in fact track the exact state of the other players and optimize their strategies accordingly.
As an alternative, mean field equilibrium (MFE) has received extensive attention recently. In an MFE, each player optimizes her expected discounted payoff, assuming that the distribution of the other players’ states is fixed. Given players’ strategy, the distribution of players’ states is an invariant distribution of the stochastic process that governs the states’ dynamics. As a solution concept for stochastic games, MFE offers several advantages over MPE. First, because players only condition their strategies on their own state (competitors’ state is assumed fixed), MFE is computationally tractable. Second, as several of the papers we cite below prove, due to averaging effects MFE provides accurate approximations to optimal behavior as the number of players grow. As a result, it provides an appealing behavioral model in games with many players.
MFE models have many applications in economics, operations research, and optimal control; e.g., studies of anonymous sequential games (Jovanovic and Rosenthal, 1988), Nash certainty equivalence control (Huang et al., 2006), continuous-time mean field models (Lasry and Lions, 2007), dynamic user equilibrium (Friesz et al., 1993), dynamic search models (Duffie et al., 2009), auction theory (Iyer et al. (2014) and Balseiro et al. (2015)), dynamic oligopoly models (Weintraub et al. (2008) and Adlakha et al. (2015)), heterogeneous agent macro models (Hopenhayn (1992) and Bewley-Hugget-Aiyagari models like in Heathcote et al. (2009)), matching markets (Arnosti et al. (2014) and Kanoria and Saban (2017)
), and evolutionary game theory(Tembine et al., 2009).
We provide three main contributions regarding MFE. First, we provide conditions that ensure the uniqueness of an MFE. This novel result is important because it implies sharp counterfactual predictions. Second, we generalize previous existence results to a general state space setting. Our existence result includes the case of a countable state space and a countable number of players, as well as the case of a continuous state space and a continuum of players. In addition, we provide novel comparative statics results for stochastic games that do not exhibit strategic complementarities.
We apply our results to well-known dynamic oligopoly models in which individual states represent the firms’ ability to compete in the market (Doraszelski and Pakes (2007)). MFE and the related concept of oblivious equilibrium have been previously used to analyze such models.444For example, Adlakha et al. (2015) uses MFE, that they call stationary equilibrium motivated by the term introduced by Hopenhayn (1992), to study models with infinite numbers of firms. Weintraub et al. (2008) introduces oblivious equilibrium to study settings with finite numbers of firms. In the models we study, larger individual states are more profitable and larger competitors’ states are less profitable. This structure is quite natural in dynamic models of competition, and we leverage it to prove our uniqueness result. We also apply our results to commonly used heterogeneous agent macroeconomic models (Heathcote et al. (2009)).
We now explain our contributions in more detail and compare them to previous work on MFE.
Uniqueness. We do not know of any general uniqueness result regarding MFE in discrete-time mean field equilibrium models. Only a few works have obtained uniqueness results in specific applications. Hopenhayn (1992) proves the uniqueness of an MFE in a specific dynamic competition model. Light (2017)
proves the uniqueness of an MFE in a Bewley-Aiyagari model under specific conditions on the model’s primitives. Our main theorem in this paper is a novel result that provides conditions ensuring uniqueness of MFE for broader classes of models. Informally, under mild additional technical conditions, we show that if the probability that a player reaches a higher state in the next period is decreasing in the other players’ states, and is increasing in the player’s own state in the current period, then the MFE is unique (see Theorem1). Hence, the conditions reduce the difficulty of showing that a stochastic game has a unique MFE to proving properties of the players’ optimal strategies (see Sections 4 and 5). In many applications, one can show that these properties of the optimal strategies arise naturally. For example, in several dynamic oligopoly models, a higher firm’s state (e.g., the quality of the firm’s product) implies higher profitability, and the firm can invest each period to improve its state. In this setting, one can show that a firm invests less when its competitors’ states are higher; hence, competitors’ higher states induce a lower state for the firm in the next period. In contrast, if the firm’s own current state is higher, it induces a higher state in the next period.
We apply our uniqueness result in a general class of dynamic oligopoly models and heterogeneous agent macroeconomic models for which MFE have been used to perform counterfactual predictions implied by a policy or system change. In the past, in the absence of this result, previous work mostly focused on a particular MFE selected by a given algorithm, or one with a specific structure. In the absence of uniqueness, the predictions often depend on the choice of the MFE, and therefore, uniqueness significantly sharpens such counterfactual analysis. We also show that the uniqueness results proved in Hopenhayn (1992) and Light (2017) can be obtained using our approach.
Existence. Prior literature has considered the existence of equilibria in stochastic games. Some prior work considered the existence of Markov perfect equilibria (MPE) (see Doraszelski and Satterthwaite (2010) and He and Sun (2017)). Adlakha et al. (2015) prove the existence of an MFE for the case of a countable and unbounded state space. Acemoglu and Jensen (2015) consider a closely related notion of equilibrium that is called stationary equilibrium and prove its existence for the case of a compact state space and a specific transition dynamic that is commonly used in economics (see Stokey and Lucas (1989)). Stationary equilibrium in the sense of Acemoglu and Jensen (2015) is an MFE where the players’ payoff functions depend on the other players’ states through an aggregator.555Acemoglu and Jensen (2015) consider a model with ex-ante heterogeneous agents. We consider a model with ex-ante identical agents. Our existence result applies for a general compact state space, more general dependence on the payoff function, and more general transitions. In this sense, it is more closely related to the result of Adlakha and Johari (2013). Adlakha and Johari (2013) prove the existence of an MFE for the case of a compact state space in stochastic games with strategic complementarities using a lattice-theoretical approach. Instead, we do not assume strategic complementarities and our state space can be any compact separable metric space. For our existence result, we assume the standard continuity conditions on model primitives that are assumed in the papers mentioned above. In addition, we assume that the optimal stationary strategy of the players is single-valued.666Since we consider only pure strategies, if the optimal stationary strategy of the players is not single-valued then the MFE operator may not be convex-valued. Similarly, problems arise in proving the existence of a pure-strategy Nash equilibrium. When the game has strategic complementarities, one can overcome this difficulty by using a lattice-theoretical approach. Concavity conditions on the profit function and the transition function can be imposed in order to ensure that the optimal stationary strategy is indeed single-valued. The main technical difficulty in proving existence is to prove the weak continuity of the nonlinear MFE operator (see Theorem 2).
Comparative statics. While some papers contain certain specific results on how equilibria change with the parameters of the model (for example, see Hopenhayn (1992) and Aiyagari (1994)), only a few papers have obtained general comparative results in large dynamic economies (see Acemoglu and Jensen (2015) for a discussion of the difficulties associated with deriving such results). Two notable exceptions are Acemoglu and Jensen (2015) and Adlakha and Johari (2013). Adlakha and Johari (2013) use the techniques for comparing equilibria developed in Milgrom and Roberts (1994) to derive general comparative statics results, and essentially rely on results about the monotonicity of fixed points. The direct application of these results requires that the MFE operator (see Equation (1)) be increasing. Our comparative statics results are different because they rely on the uniqueness of an MFE. In particular, the MFE operator is not increasing in our setting (see more details in Section 3). In this sense, our comparative static results are more similar to Acemoglu and Jensen (2015); however, our model has more general dynamics that include, for example, investment decisions with random outcomes that are typically considered in dynamic oligopoly models. Our results are useful because they establish the directional changes of MFE when important model parameters, such as the discount factor and the investment cost, change.
2 The Model
In this section we define our general model of a stochastic game and define mean field equilibrium (MFE). The model and definition of MFE are similar to Adlakha et al. (2015) and Adlakha and Johari (2013).
2.1 Stochastic Game Model
In this section we describe our stochastic game model. Differently to standard stochastic games in the literature (see Shapley (1953)), in our model, every player has an individual state. Players are coupled through their payoffs and state transition dynamics. A stochastic game has the following elements:
Time. The game is played in discrete time. We index time periods by
Players. There are players in the game. We use to denote a particular player.
States. The state of player at time is denoted by where is a separable metric space. We denote the state of all players at time by and the state of all players except player at time by .
Actions. The action taken by player at time is denoted by where . We use to denote the action of all players at time . The set of feasible actions for a player in state is given by .
States’ dynamics. The state of a player evolves in a Markov fashion. Formally, let denote the history up to time . Conditional on , players’ states at time are independent of each other. This assumption implies that random shocks are idiosyncratic, ruling out aggregate random shocks that are common to all players. Player ’s state at time depends on the past history only through the state of player at time , ; the states of other players at time , ; and the action taken by player at time , .
If player ’s state at time is , the player takes an action at time , the states of the other players at time are , and is player ’s realized idiosyncratic random shock at time , then player ’s next period’s state is given by
We assume that
is a random variable that takes valueswith probability for . is the transition function.
Payoff. In a given time period, if the state of player is , the state of the other players is , and the action taken by player is , then the single period payoff to player is .
Discount factor. The players discount their future payoff by a discount factor . Thus, a player ’s infinite horizon payoff is given by: .
In many games, coupling between players is independent of the identity of the players. This notion of anonymity captures scenarios where the interaction between players is via aggregate information about the state (see Jovanovic and Rosenthal (1988)). Let denote the fraction of players excluding player that have their state as at time . That is,
where is the indicator function of the set . We refer to as the population state of time (from player ’s point of view).
(Anonymous stochastic game). A stochastic game is called an anonymous stochastic game if the payoff function and the transition function depend on only through . In an abuse of notation, we write for the payoff to player , and for the transition function for player .
For the remainder of the paper, we focus our attention on anonymous stochastic games. For ease of notation, we often drop the subscripts and and denote a generic transition function by and a generic payoff function by where represents the population state of players other than the player under consideration. Anonymity requires that a player’s single period payoff and transition function depend on the states of other players via their empirical distribution over the state space, and not on their specific identify. In anonymous stochastic games the functional form of the payoff function and transition function are the same, regardless of the number of players .777Our results generalize for models in which the primitives depend on the number of players . In that sense, we often interpret the profit function as representing a limiting regime (see Section 4 for more details) in which the number of players is infinite.
We let be the set of all possible population states on , that is is the set of all probability measures on . We endow with the weak topology. Since is metrizable, the weak topology on is determined by weak convergence (for details see Aliprantis and Border (2006)). We say that converges weakly to if for all bounded and continuous functions we have
For the rest of the paper, we assume the following conditions on the primitives of the model:
(i) is bounded and (jointly) continuous. is continuous.888 Recall that we endow with the weak topology.
(ii) is compact.
(iii) The correspondence is compact-valued and continuous.999 By continuous we mean both upper hemicontinuous and lower hemicontinuous.
2.2 Mean Field Equilibrium
In Markov perfect equilibrium (MPE), agents’ strategies are functions of the population state. However, MPE quickly becomes intractable as the number of players grows, because the number of possible population states becomes too large. Instead, in a game with a large number of players, we might expect that idiosyncratic fluctuations of players’ states “average out”, and hence the actual population state remains roughly constant over time. Because the effect of other players on a single player’s payoff and transition function is only via the population state, it is intuitive that, as the number of players increases, a single player’s effect on the outcome of the game is negligible. Based on this intuition, related schemes for approximating Markov perfect equilibrium (MPE) have been proposed in different application domains via a solution concept we call mean field equilibrium (MFE).
Informally, an MFE is a strategy for the players and a population state such that: (1) Each player optimizes her expected discounted payoff assuming that this population state is fixed; and (2) Given the players’ strategy, the fixed population state is an invariant distribution of the states’ dynamics. The interpretation is that a single player conjectures the population state to be . Therefore, in determining her future expected payoff stream, a player considers a payoff function and a transition function evaluated at the fixed population state . In MFE, the conjectured is the correct one given the strategies being played. MFE alleviates the complexity of MPE, because in the former the population state is fixed, while in the latter players keep track of the exact evolution of the population state. We refer the reader to the papers cited in Section 1 for a more detailed motivation and rigorous justifications for using MFE.
Let . For a fixed population state, a nonrandomized strategy is a sequence of (Borel) measurable functions such that and for all . That is, a strategy assigns a feasible action to every finite string of states. Note that a single player’s strategy depends only on her own history of states and does not depend on the population state. This strategy is called an oblivious strategy (see Weintraub et al. (2008) and Adlakha et al. (2015)).
For each initial state and long run average population state , a strategy induces a probability measure over the space .101010 The probability measure on is uniquely defined (see for example Bertsekas and Shreve (1978)). We denote the expectation with respect to that probability measure by , and the associated states-actions stochastic process by .
When a player uses a strategy , the population state is fixed at , and the initial state is , then the player’s expected present discounted value is
That is, is the maximal expected payoff that the player can achieve when the initial state is and the population state is fixed at . We call the value function and a strategy attaining it optimal.
Standard dynamic programming arguments (see Bertsekas and Shreve (1978)) show that the value function satisfies the Bellman equation:
Note that the population state is fixed. In addition, under Assumption 1, there exists an optimal stationary Markov strategy. Let be the optimal stationary strategy correspondence, i.e.,
Let be the Borel -algebra on . For a strategy and a fixed population state , the probability that player ’s next period’s state will lie in a set , given that her current state is and she takes the action , is:
Now suppose that the population state is , and all players use a stationary strategy . Because of averaging effects, we expect that if the number of players is large, then the long run population state should in fact be an invariant distribution of the Markov kernel on that describes the dynamics of an individual player.
We can now define an MFE. In an MFE, every player conjectures that is the fixed long run population state and plays according to a stationary strategy . On the other hand, if every agent plays according to when the population state is , then the long run population state of all players should constitute an invariant distribution of .
A stationary strategy and a population state constitute an MFE if the following two conditions hold:
1. Optimality: is optimal given , i.e., .
2. Consistency: is an invariant distribution of . That is,
for all , where we take Lebesgue integral with respect to the measure .
Under Assumption 1 it can be shown that is nonempty, compact-valued and upper hemicontinuous. The proof is a standard application of the maximum theorem. We provide the proof for completeness (see Lemma 2 in the Appendix). In Theorem 2 we prove the existence of a population state that satisfies the consistency requirement in Definition 2.
3 Main Results
In this section we present our main results. In Section 3.1 we provide conditions that ensure the uniqueness of an MFE. In Section 3.2 we prove the existence of an MFE. In Section 3.3 we provide conditions that ensure unambiguous comparative statics results regarding MFE.
3.1 The Uniqueness of an MFE
In this section we present our uniqueness result. If there are multiple optimal stationary strategies, then each optimal stationary strategy will induce a different population state. In this case, we cannot show the uniqueness of an MFE. Thus, in order to prove uniqueness we will assume that is single-valued. For the rest of the section we will assume that is the unique selection from . In the next section we provide conditions that ensure that is single-valued (see Lemma 1).
We recall that a stationary strategy-population state pair is an MFE if and only if is optimal and is a fixed point of the operator defined by
for all . We omit the reference to in , i.e., we write instead of .
We prove uniqueness by showing that the operator has a unique fixed point. being single-valued and Theorem 2 (see Section 3.2) imply that has at least one fixed point. In Theorem 1 we will show that under certain conditions the operator has at most one fixed point.
Since the Markov kernel depends on , it is complicated to work directly with the operator . Thus, to prove the uniqueness of an MFE and to prove our comparative statics results, we introduce an auxiliary operator that is easier to work with.
For each , define the operator by
The interpretation of the operator is as follows: If the current population state is but the players conjecture that the actual population state is , and thus play according to the optimal stationary strategy , then the next period’s population state is .
We introduce the following useful definition.
We say that is -ergodic if the following two conditions hold:
(i) For any , the operator has a unique fixed point .
(ii) converges weakly to for any probability measure .
Note that is an MFE if and only if is a fixed point of the operator .
-ergodicity means that for every population state the players’ long-run state is independent of the initial state. The -ergodicity of
can be established using standard results from the theory of Markov chains in general state spaces (seeMeyn and Tweedie (2012)). When is increasing in , which we assume in order to prove the uniqueness of an MFE (see Assumption 2), then the -ergodicity of can be established using results from the theory of monotone Markov chains. These results usually require a splitting condition (see Bhattacharya and Lee (1988) and Hopenhayn and Prescott (1992)) that typically holds in applications of interest. Specifically, in Sections 4 and 5 we will show that -ergodicity holds in an important class of dynamic oligopoly models and of heterogeneous agent macro models.
We now introduce other notation and definitions that are helpful in proving uniqueness.
We assume that is endowed with a closed partial order . In the important case , we write if for each . Let . We say that a function is increasing if whenever and we say that is strictly increasing if whenever .
For we say that stochastically dominates and we write if for every increasing function we have
In addition, for the rest of the section we will assume that there exists a binary relation on , such that (i.e., and ) implies for all and for all .
Note that such binary relation always exists, for example one can take . For our uniqueness result we will further require that the the binary relation on is complete, that is, for all we either have or . In many applications (see Section 4 and Section 5) there exists a function such that and , where is continuous and increasing with respect to the stochastic dominance order . In this case, a natural complete order on arises by defining if and only if . Below, we also discuss the case of a non-complete order. We say that agrees with if for any , implies .
We say that is increasing in if for each , whenever . In addition, we say that is decreasing in if for each , whenever . We now state the main theorem of this paper. We show that if is -ergodic, is increasing in and decreasing in , and is complete and agrees with , then if an MFE exists, it is unique.
Intuitively, decreasing in implies that the probability that a player will move to a higher state in the next period is decreasing in the current period’s population state. If there are two MFEs, and , such that (i.e., is “higher” than ), then the probability of moving to a higher state under is lower than under , which is not consistent with the definition of an MFE, , and the fact that agrees with .
(i) is -ergodic. is increasing in and decreasing in .
(ii) agrees with .
(iii) is single-valued.
Suppose that Assumption 2 holds. If the binary relation is complete, then if an MFE exists, it is unique.
Proof. Let and assume that . Let be an upper set and let be two MFEs such that . We have
Thus, for any upper set we have which implies that . The first inequality follows from the fact that is decreasing in for an upper set and all . The second inequality follows from the fact that and is increasing in for an upper set and any .
We conclude that for all . being -ergodic implies that converges weakly111111 Recall that is the unique fixed point of and that is an MFE if and only if . to . Since is a closed order, we have .
We conclude that if and are two MFEs such that , then . Since agrees with , we have . That is, , which implies that and . Thus, under the players play according to the same strategy as under (i.e., for all ). We conclude that for all and . -ergodicity of implies that and have a unique fixed point. Thus, , i.e., . Similarly, we can show that implies that .
Since is complete if and are two MFEs we have or . Thus, we proved that if and are two MFEs then . We conclude that if an MFE exists, it is unique.
The assumptions on in Theorem 1 involve assumptions on the optimal strategy . Thus, these assumptions are not over the primitives of the model. In Section 4 we introduce conditions on the primitives of dynamic oligopoly models that guarantee the uniqueness of an MFE. In particular, we show that the monotonicity conditions over arise naturally in important classes of these models. In Section 5 we show how a slight modification of our uniqueness result can be applied to prove the uniqueness of an MFE in heterogeneous agent macro models.
In some applications the assumption that the binary relation is complete is restrictive. In the case that is not complete and Assumption 2 holds, the following Corollary shows that the MFEs are not comparable by the binary relation . This Corollary can be used to derive properties on the MFE when there are multiple MFEs. For example, suppose that there exist two functions , such that and , where is continuous and increasing with respect to the stochastic dominance order . We can define an order on by defining if and . Clearly, this is not a complete order. The following Corollary provides conditions that imply that if and are two MFEs, then it cannot be the case that and . We write if and .
Suppose that Assumption 2 holds. If and are two MFEs then and .
Proof. Suppose, in contradiction, that . The argument in the proof of Theorem 1 implies that . Since agrees with , we have , which is a contradiction. We conclude that . Similarly, we can show that .
3.2 The Existence of an MFE
In this section we study the existence of an MFE. We show that if is single-valued, then the operator defined in Equation (1) has a fixed point and thus, there exists an MFE. The Appendix contains the proofs not presented in the main text.
Assume that is single-valued. There exists a mean field equilibrium.
Note that can be any compact separable metric space in the proof of Theorem 2, so the existence result holds for the important cases of finite state spaces, countable state spaces, and . In addition, the proof of existence does not depend on the number of players in the game; the number of players in the game can be finite, countable or uncountable. Finally, we note that we do not require -ergodicity (see Definition 3) to show existence; instead we use compactness and continuity (see Assumption 1). The main challenge to prove existence is to prove the weak continuity of the nonlinear MFE operator. To do so, we leverage a generalized version of the bounded convergence theorem by Serfozo (1982).
We now provide conditions over the model primitives that guarantee that is single-valued when is a convex set in . Similar conditions have been used in dynamic oligopoly models.121212For similar results in a countable state space setting see Adlakha et al. (2015) and Doraszelski and Satterthwaite (2010)).
Suppose that and is convex.
(i) Assume that is concave in , strictly concave in and increasing in for each .
(ii) Assume that is increasing in and concave in for each .
(iii) is convex-valued and increasing in the sense that implies .
The following Lemma shows that the preceding conditions on the primitives of the model ensure that is single-valued.
Suppose that Assumption 3 holds. Then is single-valued.
The previous results can be summarized by the following Corollary that imposes conditions over the primitives of the model which guarantee the existence of an MFE.
Suppose that Assumption 3 holds. Then, there exists an MFE.
3.3 Comparative Statics
In this section we derive comparative statics results. Let be a partially ordered set that influences the players’ optimal decisions. We denote a generic element in by . For example, can be the discount factor, a parameter that influences the players’ payoff functions, or a parameter that influences the players’ dynamics. Throughout this section we slightly abuse notation and when the parameter influences the players’ optimal decisions we add it as a parameter. For instance, we write instead of . We say that is increasing in if for all , , and all such that . We prove that under the assumptions of Theorem 1, if is increasing in then implies that the unique MFE under is higher than the unique MFE under with respect to .
Adlakha and Johari (2013) derive comparative statics results relating to MFE in the case that is increasing in , and . They prove that implies where is the maximal MFE with respect to under . Adlakha and Johari (2013) use the techniques to compare equilibria developed in Milgrom and Roberts (1994) (see also Topkis (2011)). We note that under the assumptions of Theorem 1, is increasing in but decreasing in . Thus, the results in Adlakha and Johari (2013) do not apply to our setting. However, with the help of the uniqueness of an MFE, we derive a general comparative statics result.
We note that our comparative statics result is with respect to the order and not with respect to the usual stochastic dominance order. The machinery mentioned in the paragraph above is not directly applicable in our models, and without it we believe that comparative statics results with respect to the usual stochastic dominance order are much harder to obtain. We discuss the usefulness of our comparative static result with respect to the order in the context of dynamic oligopoly models below.
4 Dynamic Oligopoly Models
In this section we consider dynamic oligopoly models, which have received significant attention in the recent industrial organization literature (see Doraszelski and Pakes (2007) for a survey). In these models, firms’ states correspond to a variable that affects their profits. For example, the state can be the quality of the firm’s product. Per-period profits are based on a static competition game that depends on the heterogeneous firms’ state variables. Firms take actions in order to improve their state (e.g., quality) over time. Dynamic oligopoly models capture a wide range of phenomena in the industrial organization literature. In this section we leverage our results to provide conditions under which a broad class of dynamic oligopoly models admit a unique MFE. We also show comparative statics results.
More specifically, we show that under concavity assumptions and a natural substitutability condition, the MFE is unique. The substitutability condition requires that the firms’ profit function has decreasing differences in each firm’s own state and the states of the other firms. This condition implies that the marginal profit of a firm (with respect to its own state) is decreasing in the other firms’ states and arises naturally in many of these models.
We now describe the dynamic oligopoly model we consider.
States. The state of firm at time is denoted by where and is convex. For example, the state of a firm can represent the firm’s product quality level, its current productivity level, or its capacity.
Actions. At each time , firm invests to improve its state. The investment changes the firm’s state in a stochastic fashion.
States’ dynamics. A firm’s state evolves in a Markov fashion. Let be the depreciation rate. If firm ’s state at time is , the firm takes an action at time , and is firm ’s realized idiosyncratic random shock at time , then firm ’s state in the next period is given by
where is typically an increasing function that determines the impact of investment . We assume that takes positive values , where , , . That is, there exists a positive probability for a bad shock and a positive probability for a good shock . In each period, the firm’s state is naturally depreciating, but the firm can invest to improve it. Further, the outcome of depreciation and investment is subject to an idiosyncratic random shock that, for example, could capture uncertainty in R&D or a marketing campaign. Related dynamics have been used in previous literature. Importantly, our uniqueness result for dynamic oligopoly models holds under other states’ dynamics. For example, we could also assume additive dynamics . For our results to hold we need to impose some constraints on the dynamics so that the state space remains compact. (See Assumption 4 for our multiplicative dynamics and see Adlakha et al. (2015) for the additive ones.)
Payoff. The cost of a unit of investment is .141414The investment cost could be a convex function, but linearity simplifies comparative static results in the parameter . We consider a general profit function derived from a static game. When a firm invests , the firm’s state is , and the population state is , then the firm’s single period payoff is given by . We assume that there exists a complete and transitive binary relation on such that implies that for all and . Furthermore, we assume that agrees with (cf. Section 3.1). We now provide two classic examples of profit functions that are commonly used in previous literature.
Our first example is a classic quality ladder model commonly used in the industrial organization literature, where individual states represent the quality of a firm’s product (see, e.g., Pakes and McGuire (1994) and Ericson and Pakes (1995)
). Consider price competition under a logit demand system. There areconsumers in the market. The utility of consumer from consuming the good produced by firm at period is given by
where , is the price of the good produced by firm , is the consumer’s income, is the quality of the good produced by firm , and are i.i.d Gumbel random variables that represent unobserved characteristics for each consumer-good pair.
There are firms in the market and the marginal production cost is constant and the same across firms. There is a unique Nash equilibrium in pure strategies of the pricing game (see Caplin and Nalebuff (1991)). The limiting profit function can be obtained from the asymptotic regime in which the number of consumers and the number of firms grow to infinity at the same rate. The limiting profit function corresponds to a logit model of monopolistic competition given by (see Besanko et al. (1990)):
where is a constant that depends on the limiting equilibrium price, the marginal production cost, the consumer’s income, and . We define if and only if . It is easy to see that agrees with .
The second example is based on the quantity competition model of Besanko and Doraszelski (2004). We consider an industry with homogeneous products, where each firm’s state variable determines its production capacity. If the firm’s state is , then its capacity is where is an increasing, continuously differentiable, concave, and bounded function. In each period, firms compete in a capacity-constrained quantity setting game. The inverse demand function is given by , where represents the total industry output and is decreasing and continuous. For simplicity, we assume the marginal costs of all the firms are equal to zero. Given the total quantity produced by its competitors , the profit maximization problem for firm is given by . The limiting profit function can be obtained from the asymptotic regime in which firms assume they do not have market power, that is, they take as fixed. In this case, each firm produces at full capacity and the limiting profit function is given by (see also Ifrach and Weintraub (2017)):
We define if and only if . Since is increasing, agrees with .
4.1 Uniqueness and Existence of MFE in Dynamic Oligopoly Models
In this section we present our main result regarding the uniqueness (and existence) of an MFE in dynamic oligopoly models.
We introduce the following conditions on the primitives of the model. It is simple to verify that both of our dynamic oligopoly models introduced above satisfy these assumptions. We believe the conditions are quite natural so that other commonly used dynamic oligopoly models may satisfy them as well.
Recall that a function is said to have decreasing differences in on if for all and we have is said to have increasing differences if has decreasing differences.
(i) is jointly continuous. Further, it is concave and continuously differentiable in , for each .
(ii) In addition, has decreasing differences in .
(iii) is strictly concave, continuously differentiable, strictly increasing and .151515 The differentiability assumptions can be relaxed. We assume differentiability of and in order to simplify the proof of Theorem 4.
The proof of our uniqueness result for dynamic oligopoly models consists on showing that Assumption 4 together with the general assumptions of our dynamic oligopoly model imply Assumptions 1, 2, and that is a complete order. These are the conditions we used to show the existence of a unique MFE in Sections 3.1 and 3.2.
Specifically, similarly to Lemma 1, one can show that the concavity assumptions in Assumption 4 imply that is single-valued. The assumption that is used to prevent the pathological case that the Dirac measure on the point is an invariant distribution of which could violate -ergodicity (see Section 3.1). In addition, condition (iv) controls the growth of firms so that one can show that the state space remains compact. We believe our results holds with a milder version of this assumption. With this, the only remaining assumption to show holds for our dynamic oligopoly model is Assumption 2.(i). For this, we use that the profit function has decreasing differences in the state and the population state . This assumption implies that firms invest less when the population state is higher (see Lemma 3). We use this fact to show the desired monotonicity of .
Our main result for dynamic oligopoly models is the following:
Suppose that Assumption 4 holds. Then there exists a unique MFE for the dynamic oligopoly model.
4.2 Comparative Statics Results in Dynamic Oligopoly Models
Under Assumption 4 we can also derive comparative statics results for our dynamic oligopoly models. In particular, we show that an increase in the cost of a unit of investment decreases the unique MFE population state. Note that an increase in the investment cost decreases firms incentives to invest. However, a lower population state incentivizes the firms to invest more. As a consequence, our dynamic oligopoly model does not have the properties of a supermodular game (e.g., Topkis (1979) and Milgrom and Roberts (1990)). Despite of this, relying on the uniqueness of an MFE and Theorem 3 we are able to show that in fact the unique MFE decreases when the cost of a unit of investment increases.
We also derive comparative statics results regarding a change in a parameter that influences the profit function and a change in the discount factor. We show that if there is a parameter such that the marginal profit of the firms is decreasing in that parameter, then the unique MFE decreases in the parameter . For example, in the logit competition model, as marginal cost of production goes up, the unique MFE decreases. In the capacity competition model, as the potential market size increases, the MFE increases. In addition, we show that an increase in the discount factor increases the unique MFE.
We note that all of our comparative statics results are with respect to the order and not with respect to the usual stochastic dominance order as one would typically obtain using supermodularity arguments. We believe these results provide helpful information because the order relates to the single-period profit function, and therefore, MFE can be ordered in terms of firms’ payoffs. Further, typically orders a variable of economic interest, such as the average capacity level in the capacity competition model or the average quality level in the logit model.
Suppose that Assumption 4 holds. We denote by the unique MFE when the parameter that influences the firms’ decisions is .
(i) If the cost of a unit of investment increases, then the unique MFE decreases, i.e., implies .
(ii) Let be a parameter that influences the firms’ profit function. If the profit function has decreasing differences in , then the unique MFE decreases in , i.e., implies .
(iii) Assume that is increasing in . If the discount factor increases, then the unique MFE increases, i.e., implies .
5 Heterogeneous Agent Macroeconomic Models
In this section we consider heterogeneous agent macro models. In these models, there is a continuum of agents facing idiosyncratic risks only (and no aggregate risks). The heterogeneous agents make decisions given certain market prices (in Aiyagari (1994), for example, the market prices are the interest rate and the wage rate). The market prices and are determined by the aggregate decisions of all the agents in the market. We consider a setting similar to the one presented in Acemoglu and Jensen (2015). We note that this setting encompasses many important models in the economics literature. Examples include Bewley-Huggett-Aiyagari models (see Bewley (1986), Huggett (1993), and Aiyagari (1994)), models of industry equilibrium (see Hopenhayn (1992)), and models of capital accumulation and international trade (see Ventura (1997)). While Acemoglu and Jensen (2015) derive important existence and comparative statics results for these models, to the best of our knowledge there are no general uniqueness results. In this Section we show that if the agents’ strategy is decreasing in the aggregator (in the sense of Acemoglu and Jensen (2015)), there exists a unique equilibrium.
We now describe our specific model.
States. The state of player at time is denoted by where and . For example, in Bewley models typically represents agent ’s savings at period and represents agent ’s income or labor productivity at period (in this case ).
Actions. At each time , player chooses an action .
States’ dynamics. The state of a player evolves in a Markovian fashion. If player ’s state at time is , player takes an action at time , and is player ’s realized idiosyncratic random shock at time , then player ’s state in the next period is given by
where . For example, in Bewley models, in each period agents choose how much to save for future consumption and how much to consume in the current period. The agents’ labor productivity evolves exogenously and the labor productivity function determines the next period’s labor productivity given the current labor productivity. So if an agent chooses to save , is the realized random shock, and her current labor productivity is , then the agent’s next period state (savings-labor productivity pair) is given by .
Payoff. As in Acemoglu and Jensen (2015), we assume that the payoff function depends on the population state through an aggregator. That is, if the population state is , then the aggregator is given by where is a continuous function. If the aggregator is , the player’s state is , and the player takes an action , then the player’s single period payoff function is given by .
We define a complete and transitive binary relation on by if and only if . We assume that agrees with . This assumption holds in most of the heterogeneous agent macro models, where is usually assumed to be increasing with respect to first order stochastic dominance (see Acemoglu and Jensen (2015)).
Note that under the states’ dynamics defined above, and assuming that is the optimal stationary strategy, the transition kernel is given by
where and . We note that we cannot apply our uniqueness result to this model directly, since in most applications the optimal stationary strategy is not increasing in , and thus may not be increasing in . However, in most applications (for example, all the applications discussed in Acemoglu and Jensen (2015)) is increasing in .
We now show that the model has a unique MFE if the optimal strategy is decreasing in the aggregator, i.e., if implies , is -ergodic, and is increasing in . These conditions are weaker than the conditions in Assumption 2, since is not necessarily increasing in . However, using a similar argument to Theorem 1 and using the special structure of the dynamics we are able to show that the heterogeneous agent macro model has a unique MFE under the conditions stated above.161616 Note that an MFE is usually called a stationary equilibrium in the economics literature (e.g., Acemoglu and Jensen (2015)).
Assume that is single-valued, is -ergodic, and is increasing in and decreasing in the aggregator. Then the heterogeneous agent macro model has a unique MFE.
In most applications, the payoff function has increasing differences in which ensures that is increasing in . The condition that is -ergodic also usually holds in applications. For example, Huggett (1993) proves that is -ergodic in his model. Thus, in many applications, in order to ensure uniqueness, one only needs to check that is decreasing in the aggregator. In the next section we illustrate this point in a Bewley-type model introduced in Aiyagari (1994).
5.1 A Bewley-Aiyagari Model
Bewley models are widely studied and used in the modern macroeconomics literature (for a survey see Heathcote et al. (2009)). As previously mentioned, In Bewley models agents receive a state-dependent income in each period and they solve an infinite horizon consumption-savings problem; that is, the agents must decide how much to save and how much to consume in each period. The agents can transfer assets from one period to another only by investing in a risk-free bond, and have some borrowing limit. Aiyagari (1994) extends the Bewley model to a general equilibrium model with production. We now describe the model of Aiyagari (1994) in the setting of a mean field game.
In a Bewley-Aiyagari model, represents the agents’ savings and represents the agents’ labor productivity. represents the labor productivity function. That is, if the current labor productivity is then the next period’s labor productivity is given by with probability . If the agents’ labor productivity is then their income is given by where is the wage rate. The agents’ savings rate of return is .
In each period , the agents choose their next period’s savings level and consume where is the borrowing constraint, and is an upper bound on savings that ensures compactness.
The wage rate and the interest rate are determined in general equilibrium. In particular, with a representative firm that maximizes profits, the firm’s first order conditions imply that and , where denotes the aggregate per capita production function and is the total capital. We assume that is concave, continuously differentiable, and strictly increasing. In equilibrium we have where is an invariant savings-labor productivities distribution. That is, the the aggregate supply of savings equals the total capital.
We define and we define a complete and transitive binary relation on by if and only if . It is easy to see that agrees with (see Section 3.1).
The agents’ utility from consumption is given by a utility function which is assumed to be strictly concave and strictly increasing. If the agents choose to save then their consumption in the current period is . Thus, using the equilibrium conditions and , in a Bewley-Aiyagari model the payoff function is given by
It is easy to establish that is single-valued and that Assumption 1 holds. Thus, the existence of an equilibrium in a Bewley-Aiyagari model follows from Theorem 2.171717Some of the previous existence results rely on the -ergodicity of (e,g., Acikgoz (2018)). The proof presented in this paper shows that this condition is not needed in order to establish the existence of an equilibrium.
Under mild technical conditions on the labor productivity function and the utility function, the -ergodicity of can be proven in a similar manner to Benhabib et al. (2015) or Acikgoz (2018). It can be established also that the next period’s savings are increasing in the current period’s savings, i.e., is increasing in . Thus, to prove the uniqueness of an MFE in a Bewley-Aiyagari model, one needs to prove that is decreasing in the aggregator . In a recent paper, Light (2017) proves the uniqueness of an MFE for the special case that the agents’ utility function is in the CRRA (constant relative risk aversion) class with a relative risk aversion coefficient that is less than or equal to one, and a Cobb-Douglas aggregate production function. Under these assumptions, we can use the results in Light (2017) to show that is decreasing in the aggregator . Then, we can use Theorem 6 to prove the uniqueness of an MFE. As a note for future research, our results suggest that the result in Light (2017) could be generalized, weakening the conditions on the relative risk aversion and on the production function. With this, we believe our approach could be used to show uniqueness for a broader class of heterogeneous agent macro models.
This paper studies the existence and uniqueness of an MFE in stochastic games with a general state space. We provided conditions that ensure the uniqueness of an MFE. We also proved that there exists an MFE under continuity and concavity conditions on the primitives of the model. We showed that a general class of dynamic oligopoly models satisfy these conditions, and thus, these models have a unique MFE. Furthermore, a slight modification of our uniqueness result can be used in order to prove the existence of a unique MFE in heterogeneous agent macro models. We also derived general comparative statics results regarding the MFE and applied them to dynamic oligopoly models.
We believe that our results can be applied to other models in operations research and economics. For example, a fruitful avenue for future research may be to apply our approach to market design problems in online platforms, for which it is natural to assume a large-scale MFE limit. Further, typical questions of interest in these contexts involve the market response to platform market design choices. Hence, knowing that this response is unique and that one can predict its directional changes could significantly strengthen the analysis of these platforms.
- Acemoglu and Jensen (2015) Acemoglu, D. and M. K. Jensen (2015): “Robust Comparative Statics in Large Dynamic Economies,” Journal of Political Economy, 587–640.
- Acikgoz (2018) Acikgoz, O. (2018): “On the Existence and Uniqueness of Stationary Equilibrium in Bewley Economies with Production,” Journal of Economic Theory, 18–55.
- Adlakha and Johari (2013) Adlakha, S. and R. Johari (2013): “Mean Field Equilibrium in Dynamic Games with Strategic Complementarities,” Operations Research, 971–989.
- Adlakha et al. (2015) Adlakha, S., R. Johari, and G. Y. Weintraub (2015): “Equilibria of Dynamic Games with Many Players: Existence, Approximation, and Market Structure,” Journal of Economic Theory.
- Aiyagari (1994) Aiyagari, S. R. (1994): “Uninsured Idiosyncratic Risk and Aggregate Saving,” The Quarterly Journal of Economics, 659–684.
- Aliprantis and Border (2006) Aliprantis, C. D. and K. Border (2006): Infinite Dimensional Analysis: a hitchhiker’s guide, Springer.
- Arnosti et al. (2014) Arnosti, N., R. Johari, and Y. Kanoria (2014): “Managing Congestion in Decentralized Matching Markets,” in Proceedings of the fifteenth ACM conference on Economics and computation, ACM, 451–451.
- Balseiro et al. (2015) Balseiro, S. R., O. Besbes, and G. Y. Weintraub (2015): “Repeated Auctions with Budgets in Ad Exchanges: Approximations and Design,” Management Science, 864–884.
- Benhabib et al. (2015) Benhabib, J., A. Bisin, and S. Zhu (2015): “The Wealth Distribution in Bewley Models with Capital Income,” Journal of Economic Theory, 489–515.
- Benveniste and Scheinkman (1979) Benveniste, L. M. and J. A. Scheinkman (1979): “On the Differentiability of the Value Function in Dynamic Models of Economics,” Econometrica, 727–732.
- Bertsekas et al. (2003) Bertsekas, D., A. Nedi, and A. Ozdaglar (2003): Convex Analysis and Optimization, Athena Scientific.
- Bertsekas and Shreve (1978) Bertsekas, D. P. and S. E. Shreve (1978): Stochastic optimal control: The discrete time case, Academic Press New York.
- Besanko and Doraszelski (2004) Besanko, D. and U. Doraszelski (2004): “Capacity Dynamics and Endogenous Asymmetries in Firm Size,” RAND Journal of Economics, 23–49.
- Besanko et al. (1990) Besanko, D., M. K. Perry, and R. H. Spady (1990): “The Logit Model of Monopolistic Competition: Brand diversity,” The Journal of Industrial Economics, 397–415.
- Bewley (1986) Bewley, T. (1986): “Stationary Monetary Equilibrium with a Continuum of Independently Fluctuating Consumers,” Contributions to mathematical economics in honor of Gérard Debreu.
- Bhattacharya and Lee (1988) Bhattacharya, R. N. and O. Lee (1988): “Asymptotics of a Class of Markov Processes Which are not in General Irreducible,” The Annals of Probability, 1333–1347.
- Caplin and Nalebuff (1991) Caplin, A. and B. Nalebuff (1991): “Aggregation and Imperfect Competition: On the Existence of Equilibrium,” Econometrica, 25–59.
- Doraszelski and Pakes (2007) Doraszelski, U. and A. Pakes (2007): “A Framework for Applied Dynamic Analysis in IO,” Handbook of Industrial Organization.
- Doraszelski and Satterthwaite (2010) Doraszelski, U. and M. Satterthwaite (2010): “Computable Markov-Perfect Industry Dynamics,” The RAND Journal of Economics, 215–243.
- Duffie et al. (2009) Duffie, D., S. Malamud, and G. Manso (2009): “Information Percolation with Equilibrium Search Dynamics,” Econometrica, 1513–1574.
- Ericson and Pakes (1995) Ericson, R. and A. Pakes (1995): “Markov-Perfect Industry Dynamics: A Framework for Empirical Work,” The Review of Economic Studies, 53–82.
- Friesz et al. (1993) Friesz, T. L., D. Bernstein, T. E. Smith, R. L. Tobin, and B.-W. Wie (1993): “A Variational Inequality Formulation of the Dynamic Network User Equilibrium Problem,” Operations Research, 179–191.
- He and Sun (2017) He, W. and Y. Sun (2017): “Stationary Markov Perfect Equilibria in Discounted Stochastic Games,” Journal of Economic Theory, 35–61.
- Heathcote et al. (2009) Heathcote, J., K. Storesletten, and G. L. Violante (2009): “Quantitative Macroeconomics with Heterogeneous Households,” Annual Reviews in Economics, 319–352.
- Hopenhayn (1992) Hopenhayn, H. A. (1992): “Entry, Exit, and Firm Dynamics in Long Run Equilibrium,” Econometrica, 1127–1150.
- Hopenhayn and Prescott (1992) Hopenhayn, H. A. and E. C. Prescott (1992): “Stochastic Monotonicity and Stationary Distributions for Dynamic Economies,” Econometrica, 1387–1406.
- Huang et al. (2006) Huang, M., R. P. Malhamé, P. E. Caines, et al. (2006): “Large Population S