1. Introduction
The indirect evolutionary approach, pioneered by Güth and Yaari (1992) and Güth (1995), is the standard model for studying the evolution of preferences. This approach has emerged to understand how behavior appears inconsistent with material selfinterest, such as altruism, vengeance, punishment, fairness, and reciprocity, may be evolutionarily stable. ^{1}^{1}1See, for example, Güth and Yaari (1992), Güth (1995), Bester and Güth (1998), Huck and Oechssler (1999), and Ostrom (2000). In this model, players are characterized by preferences rather than preprogrammed actions. Players choose strategies to maximize their subjective preferences, but receive the real fitnesses defined by material payoff functions. Eventually, evolutionary selection is driven by differences in average fitness values.
The concept of static stability in almost all literature regarding the indirect evolutionary approach is built on symmetric twoplayer games played by a single population of players without identifying their positions. ^{2}^{2}2Exceptions can be found: von Widekind (2008, p. 61) gives a definition of stability for two separate populations and develops an outlook with some illustrative examples; Tu and Juang (2018) study the evolution of general preferences in multiple populations under various degrees of observability. In addition, all these stability concepts require robustness against a single mutation. However, the assumption that there is at most one mutation arising in a population may not hold true for all environments. Furthermore, it may be the case that multiple mutations can destabilize a population, whereas a single mutation cannot destabilize it (see Examples 3.5 and 4.4). This paper, by contrast, considers the possibility that there are more than one new entrants entering a population simultaneously; we formalize evolutionary robustness against multiple mutations in single and multipopulation settings, respectively.
Throughout this paper, players are randomly drawn to play a strategic game in terms of their preferences, and we allow for all general preferences to compete in each population. As in a lot of the indirect evolutionary approach literature, the stability criterion we use is consistent with the concept of a neutrally stable strategy, which means that mutants may coexist with the incumbents in a stable environment, but they do not spread. ^{3}^{3}3The concept of a neutrally stable strategy was introduced by Maynard Smith (1982). A natural question under this criterion is: what if there are more new mutations appearing in the postentry environment? Can the original incumbents avoid being eliminated again? ^{4}^{4}4Regarding standard evolutionary game theoretic models, van Veelen (2012) studies the question of how a neutral mutant affects a neutrally stable strategy.
Several mutations appearing simultaneously in a population can be considered not only as a small probability event, but also as a simple way of responding to this issue.
The key feature of the indirect evolutionary approach is that players can adjust their strategies according to their opponents if preferences are observable. It follows that an inefficient outcome can be destabilized by new entrants with appropriate preferences. Thus, every stable outcome is naturally endowed with some extra efficiency. ^{5}^{5}5When preferences are observable, the tendency towards efficiency is a general property of models based on the indirect evolutionary approach; see also Dekel et al. (2007), von Widekind (2008), and Tu and Juang (2018). This also indicates that an individual having materialist preferences may have no evolutionary advantage.
In this paper, we introduce multiple mutations to characterize the relations between the order of stability and the level of efficiency in single and multipopulation settings, respectively. A common feature is that if the number of mutations is somehow increased enough, then the forms of efficiency will evolve toward the Pareto frontier of the cooperative payoff region. The reason for this phenomenon is that the increase in the number of mutant types will increase the number of types of correlated deviations, and it may bring more benefits to mutants. Of course, The concept of multimutation stability can be used to refine the evolutionarily stable outcomes.
We proceed as follows. In Section 2, we introduce basic notation and notions that will be used throughout the paper. In Section 3, we study evolutionarily stable preferences against multiple mutations in a singlepopulation setting; we examine necessary and sufficient conditions for various orders of stability. Besides, in Section 4, we examine the above issues in a multipopulation setting.
2. Basic Notation and Notions
The set of players is denoted by , and for each we let be a finite set of actions available to player . For any finite set , let
denote the set of probability distributions over
. An element is known as a mixed strategy profile; a correlated strategy for matched players is any element in . Every mixed strategy profile can be interpreted as a correlated strategy in the following way: for all . We will call this correlated strategy the induced correlated strategy of . For each , let be the material payoff function of player . The function can be extended to the set as a multilinear function; it can also be extended to the set by taking expected values. Obviously, we have for all , where is the induced correlated strategy of .We interpret the value as the reproductive fitness that player obtains if an action profile is played. The evolutionary success or failure of a preference type is determined by its average fitness. Combining all the
material payoff functions, we get the vectorvalued material payoff function
that assigns to each action profile the tuple . This vectorvalued function can be extended to the set , or to the set , through .Let be the set of all von Neumann–Morgenstern utility functions over . For , a utility function , referred to as a preference type, is identified with a group of individuals in the th population who have such preferences and make the same decisions. A preference type is said to be indifferent if it is represented in terms of a constant utility function The preferences of individuals may not be represented by their respective material payoff functions; individuals in the same population may differ in their preference rankings of the potential outcomes. Specifically, if two players have the same preference relation but choose distinct strategies, their preferences can be represented here by two different utility functions congruent modulo a positive affine transformation. We assume that the number of preference types present in each population is finite, and the number of individuals corresponding to each present type is infinite. Then a probability distribution on , representing the distribution of preferences in one population, must have finite support.
Suppose that preferences are fully observable, and that individuals are drawn independently to play an player strategic game in terms of their preferences, which describes the strategic interactions among the matched individuals. Then fitness values are assigned to strategies according to the material payoff function. We can imagine that such a game is repeated infinitely many times, and one of the Nash equilibria is played in each of the resulting games. So the average fitness to each individual can be determined, and the principle of survival of the fittest suggests that a preference type can prevail only if the type receives the highest average fitness in its population.
The objective of this paper is to study the evolution of preferences without any restriction on the number of mutations in each population. We will formalize robustness against multiple mutations in two cases: one is the case where there is only one population and all players are randomly drawn from the population; the other is the case where there are separate populations and one player is randomly drawn from each of the populations. To investigate the effects of multiple mutations on populations, the concepts of multimutation stability in our single and multipopulation settings are consistent with those used in Dekel et al. (2007) and Tu and Juang (2018), respectively.
Our criteria should satisfy two requirements: if the population shares of mutants are small enough, the behavioral outcomes are almost unchanged, and no incumbent will be driven out. For a mutation set, we consider that it fails to invade if one of the mutant types in some population will become extinct, as suggested by Tu and Juang (2018). The reason for this is that interactions among mutants of a mutation set may look as if they cooperate with one another such that some of the mutants have fitness advantages over the incumbents at the expense of the other mutants. Such fitness advantages will disappear when the latter go extinct. Rather than examining the emergence of one particular preference type, we are interested in the evolutionarily viable outcome. We will give necessary and sufficient conditions for the existence of stable outcomes in the single and multipopulation settings, respectively.
3. The Singlepopulation Case
In this section, let individuals in each match be drawn randomly from a single infinite population. Suppose that players cannot identify their positions in the resulting game they play, and that the fitness values assigned to the adopted equilibrium depend only on the strategies being played, not on who play them. The material payoff function and players’ preferences are presented below.
For ease of presentation, only twoplayer games will be considered in the singlepopulation case. Under the above assumptions, the game that is used for the fitness assignment is a symmetric game, that is, and for any strategies and . We will write for the common set of pure strategies available to players in any position. For a preference type , the expected subjective utility to its strategy , when played against some strategy , is denoted by , where , . Let be the set of all probability distributions on having finite supports.
Assume that individuals for each match are drawn independently according to , which represents the distribution of preference types in the population. Interactions between any two players in , the support of , are characterized by the twoplayer game . Combining this with the symmetric game , the pair is referred to as an environment. A strategy for a preference type is a function ; players of the type would choose the strategy when playing against the type . Define by . ^{6}^{6}6In a symmetric game , the notation means , and the notation will be used for . We say is an equilibrium in the game if for any , ,
Let denote the set of all such equilibria in . ^{7}^{7}7The set is nonempty, because not only does a mixed Nash equilibrium always exist, but we can also find a symmetric mixed Nash equilibrium in any finite symmetric game.
For a preference distribution and an equilibrium , the pair is called a configuration
. By the law of large numbers, the
average fitness of a preference type with respect to is given byWe define the aggregate outcome of to be the correlated strategy satisfying
for all . Then the equality holds for any , , which means that the aggregate outcome is a symmetric correlated strategy, so . In particular, a symmetric strategy profile is called an aggregate outcome if the induced correlated strategy is the aggregate outcome of some configuration, where is defined by for all .
Let be the incumbent preference distribution. Suppose that there are distinct mutations introduced into the population simultaneously with population shares , respectively. In this paper, we allow for all possible types of mutants to compete, except that they must be distinguishable from the incumbents, that is, for all . The set , called an th order mutation set, is denoted by , and the multiset is denoted by . We will use the notation for the sum . The postentry distribution of preferences can be characterized by
where is the degenerate distribution concentrated at for , and its support, , is . For notational convenience, we shall often abbreviate , , and to , , and , respectively.
Since preference evolution is driven by differences in fitness values, it is necessary to assume that incumbents receive the same average fitness for a configuration to be stable.
Definition 3.1.
A configuration is said to be balanced if for every , .
To satisfy the condition that mutants cannot significantly affect the aggregate outcome if they are rare enough, we demand that the strategic interactions between incumbents will remain unchanged when they are matched again in the postentry environment.
Definition 3.2.
Let be a configuration in , and let a mutation set be given with . In , a postentry equilibrium is focal relative to if for any , . Let be the set of all focal equilibria relative to in , called a focal set.
Remark.
Note that the focal set is nonempty, and that it is independent of . In addition, the desired property that as is obviously held for any .
The criterion for multimutation stability of a configuration defined in the singlepopulation case is formulated as follows.
Definition 3.3.
In , a configuration is said to be multimutation stable of order , or th order stable, if it is balanced, and if for any and any , there exists some such that for every with , either 1 or 2 is satisfied.

[label=()]

for some and for every .

for every , .
If is stable of order , it is usually simply said to be stable. If is stable of all orders, it is said to be infinitely stable. We call a symmetric strategy profile multimutation stable of order , or th order stable, if it is the aggregate outcome of an th order stable configuration.
Remark.
There is a noteworthy difference between the stability concepts defined here and in Dekel et al. (2007). The statement of the stability condition used in Dekel et al. (2007) contains the possibility that there are both a mutant type and an incumbent type who have the lowest average fitness, but at the same time there is another incumbent type who has a higher average fitness. ^{8}^{8}8Once it happens, the dynamic process would seem to eventually lead to the extinction of both the mutant type and the incumbent type who have the lowest average fitness. In this paper, our definition directly excludes this possibility.
It is easy to see that a configuration must be kept multimutation stable of lower orders if it is a stable configuration of order greater than .
Lemma 3.4.
Let be an th order stable configuration with . Then it is stable of order for any .
Proof.
Suppose that a balanced configuration is not stable of order . This means that there exist a mutation set and an equilibrium such that for arbitrarily given , we can choose with for which the following two conditions are satisfied: (1) for every , there is with ; (2) the inequality holds for some , .
Let be a positive integer with . To prove that the configuration is not th order stable, we consider a new mutation set with satisfying for and . Suppose that the preferences of each of the mutant types are such that their strategic behavior is the same as that of the mutant type . Then there is an equilibrium such that for the given , we can choose for which for all , and for . (Here denotes the postentry distribution of preferences after also entering the population.) This clearly leads to the result that is not stable of order for . ∎
The next example shows that a stable configuration is not stable of higher order; the converse of Lemma 3.4 is not true.
Example 3.5.
Suppose that the following symmetric game characterizes the fitness assignment. The set of actions available to each player is , and the material payoff function is denoted by .
,  ,  
,  , 
Let be a monomorphic configuration constituted as follows: for , if and otherwise, and . We first show that the configuration is stable. Let be an entrant with a population share , and let be a focal equilibrium relative to . Then the average fitnesses of and are, respectively,
and
If , we have , and therefore whenever is sufficiently small. If , it is easy to check that
for any focal equilibrium and any . Thus is stable, and so is the strategy profile . However, as we see below, the configuration is not stable of order .
Let and be two distinct mutant types introduced into the population with population shares and , respectively. Suppose that the adopted equilibrium satisfies and for , . Then we obtain and for all , where . This shows that is not a second order stable configuration. Furthermore, Theorem 3.16 will guarantee that is the unique stable strategy profile, and that there exists no strategy profile which is stable of order greater than in this game.
Remark.
Because mutants entering the population can have any preference type, technically indifferent types are used instead of potential entrants that are well adapted to the environments.
In symmetric games, a specific type of efficiency focusing on symmetric strategy profiles is frequently used.
Definition 3.6.
In a symmetric game , a strategy (or a strategy profile ) is efficient if for all .
Obviously, the efficient fitness is uniquely determined, but an efficient strategy may not be unique. Except for efficiency of a symmetric strategy profile, here the concept of Pareto efficiency is needed to characterize the features of multimutation stability.
Definition 3.7.
For a nonempty subset of , the Pareto frontier of is defined as
We will focus attention on the Pareto frontiers of the noncooperative payoff region and the cooperative payoff region induced by the material payoff function. ^{9}^{9}9For a twoplayer game , the noncooperative payoff region and the cooperative payoff region refer to the plane regions and , respectively.
Remark.
In a symmetric twoplayer game, a strategy satisfying must be efficient, but not vice versa as Example 3.5 shows.
Before we derive the necessary conditions for configurations to be multimutation stable, we first introduce two useful lemmas.
Lemma 3.8.
Suppose that is a stable configuration in . Then for any , , the equality
holds for every .
Proof.
Suppose that there exist distinct types , such that
for some . Introduce a mutant type into the population with a population share . Suppose that the postentry equilibrium chosen from satisfies and for all . Then the difference between and is
Thus the inequality eventually holds whenever is sufficiently small, and so is not a stable configuration. ∎
Lemma 3.9.
If is a stable configuration in , then is efficient for every .
Proof.
Suppose that there exists such that is not efficient. Consider an entrant with a population share . Let be efficient, and suppose that the chosen equilibrium satisfies and for all . Then the difference
for any , and thus the configuration is not stable. ∎
Dekel et al. (2007) show that a stable outcome must be efficient when preferences are observable. In this paper, we demonstrate that there is a tendency for efficiency levels to increase with higher orders of stability. If the efficiency level of an outcome is not high enough, there are different types of mutants that can destabilize the configuration together based on their own preferences as follows. Mutants of any type would maintain some preentry outcomes when matched against themselves or against the incumbents, and would achieve more efficient outcomes when matched against the other mutant types.
Theorem 3.10.
Let be a configuration in . Suppose that is efficient in and that , .

[label=()]

If is stable, then .

If is stable of order , then .

If is stable of order , then .
Proof.
Let be stable in . Then by Lemma 3.8, we have and for any , . By Lemma 3.9, both and are efficient. Therefore, for any , .
For 2, assume that there are , with . Then there is a strategy profile satisfying for every and for some . Let with be introduced. Suppose that the adopted equilibrium satisfies , and the relations and hold for , and for all . For each , the difference between the average fitnesses of and can be written as
where , and hence for some , no matter what is. Therefore, is not a second order stable configuration.
For 3, we argue by contradiction. Let the configuration be stable of order in , and suppose that there exist , such that . By Lemma 3.4 and property 1, the equality is true, and thus . Because is a symmetric twoplayer game, there exist , such that
Introduce into the population with . Suppose that the equilibrium satisfies the following conditions. When two distinct types of mutants are matched,
Otherwise, for a given and for any , we have and for some . Let for , , . Then the difference between the average fitnesses of and is
for any , and thus we have arrived at a contradiction. ∎
Remark.
Furthermore, Lemma 3.4 implies that if is stable of order then , and that if is stable of order then .
When a configuration is stable, it is clear that the average fitness of every incumbent must be efficient, since all incumbents receive the efficient fitness in each of their interactions, as described in part 1 of Theorem 3.10. Besides, the aggregate outcome of a stable configuration also corresponds to the efficient fitness value.
Corollary 3.11.
Let be a stable configuration in with the aggregate outcome , and let be an efficient strategy in . Then the relations
hold for every .
In Example 3.5, the stable configuration can be invaded and displaced by a mutation set of two types of mutants. In fact, mutants’ invasion capacity cannot unrestrictedly expand with the new types entering the population. We show here that a third order stable configuration in a twoplayer environment will also be stable of all orders.
Theorem 3.12.
In , a configuration is infinitely stable if and only if it is stable of order .
Proof.
One direction follows immediately from Lemma 3.4: an infinitely stable configuration must be stable of order . For the converse, suppose that is a third order stable configuration in . Then, by the remark after Theorem 3.10, we have for all , , where is an efficient strategy. Let , and let be introduced into the population with . For any given , , and for arbitrarily chosen , the difference between their average fitnesses and is