 # The Curse of Ties in Congestion Games with Limited Lookahead

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

Consider the following situation, where two players want to travel from origin to destination in the (extension-parallel) graph on the right. They can take the metro , which takes 6 minutes, or they can take the bike and then walk either the long (but scenic) route , which takes minutes, or the short route , which takes minute. There is only one bike: if only one of them takes the bike it takes minutes; otherwise, someone has to sit on the backseat and it takes them minutes. Both players want to minimize their own travel time.

Suppose they announce their decisions sequentially. There are two possible orders: either the red player 1 moves first or the blue player 2 moves first. We consider the sequential-move version of the game where player moves first. There are three possible subgames that player 2 may end up in, for which the corresponding game trees are as follows:

A strategy for player 2 is a function that tells us which action player 2 plays given the action of player 1. Player 2 will always choose an action that minimizes his travel time and may break ties arbitrarily when being indifferent. The boldface arcs give a possible subgame-perfect strategy for player 2. If player fixes this strategy, then player 1 is strictly better off taking the bike and walking the long route (for a travel time of compared to a travel time of for the other cases). That is, the only subgame-perfect response for player 1 is . This shows is a subgame-perfect outcome. However, this outcome is rather peculiar: Why would player 1 walk the long route if his goal is to arrive as quickly as possible? In fact, the outcome is not stable, i.e., it does not correspond to a Nash equilibrium.

Subgame-perfect outcomes are introduced as a natural model for farsightedness [17, 18], or “full anticipation”, and have been studied for various types of congestion games [2, 3, 4, 17]. Another well-studied notion in this context are outcomes of greedy best-response [7, 8, 12, 20], i.e., players enter the game one after another and give a best response to the actions played already, thus playing with “no anticipation”. Fotakis et al.  proved that greedy best-response leads to stable outcomes on all series-parallel graphs (which contain extension-parallel graphs like the one above as a special case).

In fact, in the above example both and

are greedy best-response outcomes and they are stable. The example thus illustrates that full lookahead may have a negative effect on the stability of the outcomes. After a moment’s thought, we realize that in the subgame-perfect outcome the indifference of player 2 is exploited (by breaking ties accordingly) to force player 1 to play a suboptimal action. Immediate questions that arise are: Does full lookahead guarantee stable outcomes if we adjust the travel times such that the players are no longer indifferent (i.e., if we make the game

generic)? What is the lookahead that is required to guarantee stable outcomes? What about the inefficiency of these outcomes? In this paper, we address such questions.

A well-studied playing technique for chess introduced by Shannon  is to expand the game tree up to a fixed level, use an evaluation function to assign values to the leaves and then perform backward induction to decide which move to make. Based on this idea, we introduce -lookahead outcomes as outcomes that arise when every player uses such a strategy with levels of backward induction. The motivation for our studies is based on the observation that such limited lookahead strategies are played in many scenarios. In fact, there is also experimental evidence (see, e.g., ) that humans perform limited backward induction rather than behaving subgame-perfectly or myopically (i.e., ).

In general, our notion of -lookahead outcomes can be applied to any game that admits a natural evaluation function for partial outcomes (details will be provided in the full version of this paper). In this paper, we demonstrate the applicability of our novel notion by focussing on congestion games. These games admit a natural evaluation function by assigning a partial outcome its current cost, i.e., the cost it would have if the game would end at that point.

#### Our model.

We introduce -lookahead outcomes as a novel solution concept where players enter the game sequentially (according to some arbitrary order) and anticipate the effect of subsequent decisions. In a -lookahead outcome , the th player with computes a subgame-perfect outcome in the subgame induced by with players (according to the order) and chooses his corresponding action. Our model interpolates between outcomes of greedy best-response () and subgame-perfect outcomes (, the number of players). Our main goal is to understand the effect that different degrees of anticipation have on the stability and inefficiency of the resulting outcomes in congestion games.

We combine limited backward induction with the approach of Paes Leme et al.  who proposed to study the inefficiency of subgame-perfect outcomes. Subgame-perfect outcomes have several drawbacks as model for anticipating players, which are overcome (at least to some extent) by considering -lookahead outcomes instead:

1. Computational complexity. Computing a subgame-perfect outcome in a congestion game is PSPACE-complete  and it is NP-hard already for 2-player symmetric network congestion games with linear delay functions . This adds to the general discussion that if a subgame-perfect outcome cannot be computed efficiently, then its credibility as a predicting means of actual outcomes is questionable. On the other hand, computing a -lookahead outcome for constant can be done efficiently by backward induction.

2. Limited information. Due to lack of information players might be forced to perform limited backward induction. Note that in order to expand the full game tree, a player needs to know his successors, the actions that these successors can choose and their respective preferences. In practice, however, this information is often available only for the first few successors.

3. Clairvoyant tie-breaking. Players may be unable to play subgame-perfectly, unless some clairvoyant tie-breaking rule is implemented. To see this, consider the example introduced above. Note that in the subgame induced by (as well as ) player is indifferent between playing and . Thus, player 1 has no way to play subgame-perfectly with certainty: in order to do so he will need to correctly guess how player 2 is going to break ties. Such clairvoyant tie-breaking is not required for reaching a -lookahead outcome.

#### Our results.

We study the efficiency and stability of -lookahead outcomes. We call an outcome stable if it is a Nash equilibrium. In order to assess the inefficiency of -lookahead outcomes, we introduce the -Lookahead Price of Anarchy (-LPoA) which generalizes both the standard Price of Anarchy  and the Sequential Price of Anarchy  (see below for formal definitions).

Quantifying the -Lookahead Price of Anarchy is a challenging task in general. In fact, even for the Sequential Price of Anarchy (i.e., for ) no general techniques are known in the literature. In this paper, we mainly focus on characterizing when -lookahead outcomes correspond to stable outcomes. As a result, our findings enable us to characterize when the -Lookahead Price of Anarchy coincides with the Price of Anarchy. We show that this correspondence holds for congestion games that are structurally simple (i.e., symmetric congestion games on extension-parallel graphs), called simple below. Further, a common trend in our findings is that the stability of -lookahead outcomes crucially depends on whether players do not or do have to resolve ties (generic vs. non-generic games). Our main findings in this paper are as follows:

1. We show that for generic simple congestion games the set of -lookahead outcomes coincides with the set of Nash equilibria for all levels of lookahead . As a consequence, we obtain that the -LPoA coincides with the Price of Anarchy (independently of ), showing that increased anticipation does not reduce the (worst-case) inefficiency. On the other hand, we show that only full anticipation guarantees the first player the smallest cost, so that anticipation might be beneficial after all. We also show that the above equivalence does not extend beyond the class of simple congestion games. (These results are presented in Section 3.)

2. For non-generic simple congestion games, subgame-perfect outcomes my be unstable (as the introductory example shows) but we prove that they have optimal egalitarian social cost. For the more general class of series-parallel graphs, we prove that the congestion vectors of 1-lookahead outcomes coincide with those of global optima of Rosenthal’s potential function. In particular, this implies that the

-LPoA is bounded by the Price of Stability. (See Section 3.)

3. We also study cost-sharing games and consensus games (see below for definitions). For consensus games, subgame-perfect outcomes may be unstable. In contrast, if players break their ties consistently all -lookahead outcomes are optimal. Similarly, for non-generic cost-sharing games we show that even in the symmetric singleton case subgame-perfect outcomes may be unstable. For both symmetric and singleton games this can be resolved by removing the ties. We also observe a threshold effect with respect to the anticipation level. For generic symmetric cost-sharing games, we show -lookahead outcomes are stable but guaranteed to be optimal only for . For affine delay functions the -LPoA is non-increasing (i.e., the efficiency improves with the anticipation). For generic singleton cost-sharing games, -lookahead outcomes are only guaranteed to be stable for . (These results can be found in Section 4.)

#### Related work.

The idea of limited backward induction dates back to the 1950s 

and several researchers in artificial intelligence (see, e.g.,

) investigated it in a game-theoretic setting. Mirrokni et al.  introduce -lookahead equilibria that incorporate various levels of anticipation as well. Their motivation for introducing these equilibria is very similar to ours, namely to provide an accurate model for actual game play. However, their -lookahead equilibria correspond to Nash equilibria rather than greedy best-response outcomes and none of the equilibria correspond to subgame-perfect outcomes. Moreover, lookahead equilibria are not guaranteed to exist. For example, Bilo et al.  show that symmetric singleton congestion games do not always admit 2-lookahead equilibria (for the “average-case model”).

Subgame-perfect outcomes are special cases of -lookahead outcomes. Paes Leme et al.  generalize the Price of Anarchy notion to subgame-perfect outcomes and show that the Sequential Price of Anarchy can be much lower than the Price of Anarchy if the game is generic. On the other hand, this does not necessarily hold if the game is non-generic (see, e.g., [2, 3, 14]).

We formally define congestion games and introduce some standard notation. We then introduce our notion of -lookahead outcomes and the inefficiency measures studied in this paper. Finally, we comment on the impact of ties and different player orders in these games.

#### Congestion games.

A congestion game is a tuple where is a finite set of players, a finite set of resources, the action set of player , and a delay function for every resource .111We use to denote the set , where is a natural number. Unless stated otherwise, we assume that is non-decreasing. We define as the set of action profiles or outcomes of the game. Given an outcome , the congestion vector specifies the number of players picking each resource, i.e., . The cost function of player is given by . We call a congestion game symmetric if for all .

We say that is a best response to if for all .222We use the standard notation and . An outcome of a congestion game is a (pure) Nash equilibrium (NE) if for all , is a best response to . We use to denote the set of all Nash equilibria of .

An order on the players is a bijection . We denote the sequential-move version of a game with respect to order by . The outcome on the equilibrium path of a subgame-perfect equilibrium in is an action profile of and we refer to it as the subgame-perfect outcome (SPO). We use to refer to the set of all subgame-perfect outcomes (with respect to any order of the players) of a game .

Let be a congestion game and an order on the players. For , define as the congestion game with which we obtain from if only the first players (according to ) play. Let , where is the identity order (i.e., ). For notational convenience, for we set . Further, if the order is defined on a larger domain than the player set of , we define for the unique bijection satisfying iff for all

###### Definition 1.

Let be an -player congestion game and let . An action profile is a -lookahead outcome of if there exists an order on the players so that for each we have that equals the action played by player in some subgame-perfect outcome of that corresponds to the order , where is the subgame of induced by .333Note that we would need to write instead of without our assumption that for .

We say a -lookahead outcome corresponds to the order if can be used as the order in the definition above. We also define a -lookahead outcome for as an -lookahead outcome. We use to denote the set of all -lookahead outcomes of a game .

Assuming for ease of notation, is a -lookahead outcome (corresponding to the identity) if and only if is the action played by the first player in a subgame-perfect outcome (corresponding to the identity) of and is a -lookahead outcome (corresponding to the identity) in the game induced by .

###### example 2.

Let be a congestion game with , , and . Let the delay functions be given by Suppose the players enter the game in the order and all anticipate their own decision and the next player (). Player 1 then computes the unique subgame-perfect outcome depicted on the left in Figure 1. Hence he chooses the resource . This brings player 2 in the subgame whose game tree is depicted on the right in Figure 1. His unique subgame-perfect choice is . This shows that the only 2-lookahead outcome corresponding to the identity of this game is .

We introduce the -Lookahead Price of Anarchy (-LPoA) to study the efficiency of -lookahead outcomes, which generalizes both the Price of Anarchy  and the Sequential Price of Anarchy . We consider two social cost functions in this paper: Given an outcome , the utilitarian social cost is defined as ; the egalitarian social cost is given by .

Given a game , let refer to an outcome of minimum social cost. The -Lookahead Price of Anarchy (-LPoA) of a congestion game is

 k-LPoA(G)=maxA∈k-LO(G)∑i∈Nci(A)∑i∈Nci(A∗). (1)

Here the -LPoA is defined for the utilitarian social cost; it is defined analogously for the egalitarian social cost. Recall that the Price of Anarchy (PoA) and the Sequential Price of Anarchy (SPoA) refer to the same ratio as in (1) but replacing “” by “” and “”, respectively. The Price of Stability (PoS) refers to the same ratio, but minimizing over the set of all Nash equilibria.

#### Curse of ties.

When studying sequential-move versions of games, results can be quite different depending on whether or not players have to resolve ties. We next introduce two notions to avoid/resolve ties.

We first introduce the notion of a generic congestion game. Intuitively, this means that every player has a unique preference among all available actions.

###### Definition 3.

A congestion game is generic if for all , and , implies .

Note that if is generic then is also generic for every order of the players and and every induced subgame is generic as well. Hence if is the unique subgame-perfect outcome (say respect to the identity) of , then the subgame induced by is again generic, so that must be the only subgame-perfect outcome (with respect to the identity) of . With induction, it then follows that for generic games, there is a unique -lookahead outcome which is equal to the unique subgame-perfect outcome. For non-generic games, each subgame-perfect outcome is an -lookahead outcome, but the reverse may be false.

To see this, consider a congestion game with , and . Let , and . Then is a subgame-perfect outcome, correspondig to the SPE depicted to the left below, so player 1 may choose in an -lookahead outcome.

However, in the subgame induced by , whose game tree is depicted to the right, both and are subgame-perfect (player 2 is indifferent), so player 2 may respond , giving the -lookahead outcome . This is not a subgame-perfect outcome.

Rather than assuming that no ties exist, we can also restrict the definition of a subgame-perfect outcome: A tie-breaking rule for player is a total partial order on the action set . If player adopts the tie-breaking rule , then only is a best response to if In particular, ensures that player has a unique best response. As a result, there is a unique subgame-perfect outcome for each tie-breaking rule and order of the players. For symmetric congestion games, we can consider the special case of a common tie-breaking rule on that all players adopt.

#### Effect of the player order.

Whether all subgame-perfect outcomes are stable may depend on the order of the players. For example, for consensus games all subgame-perfect outcomes corresponding to a tree respecting order are stable (see Example 27). Whenever we make a claim such as “all subgame-perfect outcomes are stable”, this should be read as “subgame-perfect outcomes are stable for all orders”.

###### Theorem 4.

If is a symmetric congestion game and a -lookahead outcome with respect to , then is a -lookahead outcome with respect to for every order .

###### Proof.

Let be a -lookahead outcome with respect to . Then is the first action of some SPO with respect to the order in . Let be a subgame-perfect equilibrium inducing the SPO .

In the sequential move-version , the root node is a decision node for player and is a valid strategy for player (since the game is symmetric). Similarly, are valid strategies for . In fact, the game tree of is the same as the game tree of up to a relabelling of the players. This means that in both games we verify the same equations when determining whether is a subgame-perfect equilibrium. Hence is a subgame-perfect equilibrium in as well. In the corresponding subgame-perfect outcome of , player plays . Similar argumentation shows that in the subgame induced by , the action is the action of in a subgame-perfect outcome of . ∎

This theorem allows us to assume that the order is the identity when proving stability for symmetric games. Moreover, any permutation of a -lookahead outcome is again a -lookahead outcome, which is a useful fact that we exploit in the proofs below.

Due to lack of space, some of the proofs are omitted from the main text below and will be provided in the full version of the paper.

## 3 Symmetric network congestion games

A single-commodity network is a directed multigraph with two special vertices such that each arc is on at least one directed -path. In a symmetric network congestion game (SNCG), the common set of actions is given by the set of all directed paths in a single-commodity network . A series-parallel graph (SP-graph) either consists of (i) a single arc, or (ii) two series-parallel graphs in parallel or series. An extension-parallel graph (EP-graph) either consists of (i) a single arc, (ii) two extension-parallel graphs in parallel, or (iii) a single arc in series with an extension-parallel graph.

In our proofs we exploit the following equivalences:

###### Lemma 5 (Nested intersections property).

The common action set of each SNCG on an EP-graph satisfies the following three equivalent properties:

1. For all distinct either or .

2. For all distinct , implies .

3. There is no bad configuration (see ), i.e., for all distinct , or .

The non-trivial part that an SNCG has no bad configuration if and only if its network is extension-parallel is shown by Milchtaich .

Fotakis et al.  show that each 1-lookahead outcome is a Nash equilibrium for SNCG on SP-graphs. We prove that the converse also holds for EP-graphs.

###### Theorem 6.

For every SNCG on an EP-graph, the set of 1-lookahead outcomes coincides with the set of Nash equilibria.

###### Proof.

It remains to show that each Nash equilibrium of is a permutation of a 1-lookahead outcome corresponding to the identity.

Consider an NE with corresponding congestion vector . Let be the set of actions costing the least for the first player, that is, those minimising . This corresponds to the set of NE of the 1-player game. Assume towards contradiction no one plays an action from . Let . Since is an NE, has become more expensive as more players joined in, so some overlaps with it. Pick with maximal. Then by the nested intersection property, no intersects with . Hence for any (these are not chosen). We find

 ∑r∈P∖Ajdr(xr+1)=∑r∈P∖Ajdr(1)<∑r∈Aj∖Pdr(1)≤∑r∈Aj∖Pdr(xr)

using that and . This contradicts the fact that is an NE.

Let be a player picking an action from , so that forms a 1-lookahead outcome. Define . Suppose we have defined so that forms a 1-lookahead outcome for some . The profile

 A′′=A∖A′=(Aj)j∈σ−1{i+1,…,n}

forms an NE in the game induced by . Repeat the argument above for and to define . ∎

### 3.1 Stability and inefficiency of generic games

As shown in the introduction, SPOs are not guaranteed to be stable for SNCGs on EP-graphs. However, stability is guaranteed if the game is generic.

###### Theorem 7.

For every generic SNCG on an EP-graph, the set of subgame-perfect outcomes coincides with the set of Nash equilibria.

###### Proof.

Because the game is generic, there is a unique 1-lookahead outcome (up to permutation) and hence a unique NE: The game is generic and thus there is a unique cheapest first path . The game is generic and hence so is the subgame of induced by . Thus there is a unique cheapest second path, and so on.

We prove the statement by induction on the number of players . The claim is true for . Suppose the claim holds for all generic SNCGs on EP-graphs with less than players and let be an SNCG on an EP-graph with players. Let be an SPO and the corresponding SPE, corresponding to some order which we may assume to be the identity by relabeling the players. Given an action of player 1, prescribes an SPE in the game induced by . By our induction hypothesis (the subgame induced by has players), the SPO corresponding to is a permutation of a 1-lookahead outcome in this subgame. In particular, if is the unique 1-lookahead outcome of the game and player 1 plays , then the resulting outcome according to is a permutation of .

Suppose towards contradiction that for all . By induction is a NE in the subgame induced by (by the same argument as before). Since is not a Nash equilibrium and all players except 1 are playing a best response, there is an action so that . If player 1 switches to , then the other players are still playing a best response by [6, Lemma 1]. So is an NE, hence a permutation of . This means that for some (and some permutation of ), which yields a contradiction: , where the last inequality follows because is subgame-perfect for player 1. ∎

###### Theorem 8.

For every SNCG on an EP-graph, each Nash equilibrium is a subgame-perfect outcome.

###### Sketch.

Call game (with delay functions ) close to if for all paths and congestion vectors and

 ∑r∈Pdr(xr)<∑r∈Qdr(xr)⟹∑r∈Pd′r(xr)≤∑r∈Qd′r(xr).

Suppose is given together with a Nash equilibrium . If is a generic game close to which also has has Nash equilibrium, then by Theorem 7 this is the unique subgame-perfect outcome of . The close condition above (invented by Milchtaich ) exactly ensures is then also a subgame-perfect outcome in . It remains to find a close generic game, which can be done by adjusting the delay functions of ; each path contains a resource he shares with no other path , so we can increase the cost of individual paths without effecting the other path costs. ∎

The following theorem is the main result of this section.

###### Theorem 9.

Let be a generic SNCG on an EP-graph. Then for every the set of -lookahead outcomes coincides with the set of Nash equilibria. As a consequence, -.

###### Proof.

Since there is a unique Nash equilibrium and -lookahead outcome up to permutation, it suffices to show that each -lookahead outcome is a Nash equilibrium. By relabelling the players, we can assume that the order of the players is the identity. Let denote the unique 1-lookahead outcome of corresponding to the identity. Then has unique 1-lookahead outcome .

Let be a -lookahead outcome. Since each SPO of is a permutation of a 1-lookahead outcome, we know that . This implies that the subgame of induced by has some permutation of as unique 1-lookahead outcome (where we perform the set operations seeing the tuple as a multiset). This means that . Continuing this way, we see that for and for we find . Hence will be a permutation of . ∎

The result of Theorem 9 does not extend to series-parallel graphs.

###### Proposition 10.

For any SP-graph that is not EP, there is a generic SNCG on such that the sets of 1-lookahead and -lookahead outcomes are disjoint.

###### Proof.

Consider the single-commodity network with four arcs and -paths , , and . Each series-parallel graph that is not extension-parallel has this network as a minor (see ). Thus it suffices to give a counterexample on this network. Consider the generic three-player game with delay functions

 (dr(1),dr(2),dr(3))=(1,3,100)(ds(1),ds(2),ds(3))=(2,4,200)(dt(1),dt(2),dt(3))=(1.1,4.1,100.1)(du(1),du(2),du(3))=(2.2,3.2,100.2).

The unique 1-lookahead outcome (up to permutation) is and the unique -lookahead outcome (up to permutation) is . ∎

We next show that anticipation may still be beneficial for the first player.

###### Theorem 11.

Let be a generic SNCG on an EP-graph. Let be a subgame-perfect outcome with respect to the identity. Then . In particular, for any -lookahead outcome with respect to the identity.

###### Proof.

Let be a permutation of the unique NE of for which . Let be the unique subgame-perfect outcome with respect to the identity; then this is some permutation of by Theorem 9.

If player 1 plays , then his successors will play (not necessarily in that order), so that . Since is some permutation of and since (which may equal for some ) is the unique element from with the lowest cost in the profile , we find and . Similarly, player can ensure himself the cost in the subgame induced by and therefore . This proves

Finally, note that if is a -lookahead outcome, then is some permutation of and therefore for some . ∎

The following example shows that the cost of the first players does not decrease monotonically with his lookahead. In fact, a generalization of this example shows that only full lookahead guarantees the smallest cost for player 1.

###### example 12.

Consider the generic symmetric singleton congestion game with , and . The subgame-perfect outcome with respect to the identity is if is even and if

is odd. Thus, if

In contrast to the above, the first player is not guaranteed to achieve minimum cost with full lookahead if the game is non-generic.

###### example 13.

Suppose a symmetric singleton congestion game with delay functions is played by an odd number of players. The successors of player 1 can decide which resource becomes the most expensive one and enforce that this is the one that player 1 picks, so that player 1 is always worse off.

### 3.2 Inefficiency of non-generic games

Let be an SNCG and let be an outcome. Let be the corresponding congestion vector. We define the opportunity cost of in as the minimal cost that a new player entering the game would have to pay, i.e., ; this definition is implicit in [8, 20]. The worst cost of in is the egalitarian social cost, i.e., .

###### Lemma 14.

Let be an SNCG on a SP-graph with players and let be the same game with players. For any action profile in and any 1-lookahead outcome of , we have . Further, if , then .

The lemma above can be proved by induction on the graph structure.

###### Corollary 15.

For SNCGs on SP-graphs, all 1-lookahead outcomes have the same value for the potential function.

###### Proof.

Let and be 1-lookahead outcomes (with respect to the identity). If , then since both are chosen greedily we must have . For , the profiles and are also 1-lookahead outcomes, so that by an inductional argument and using that 1-lookahead outcomes have the same opportunity costs (Lemma 14), we find

 Φ(A)=cn(A)+Φ(A′)=OA′+Φ(A′)=OB′+Φ(B′)=Φ(B).

Applying induction on the graph structure again, we are able to derive that each 1-lookahead outcome is a global optimum of Rosenthal’s potential function.

###### Proposition 16.

Let be an SNCG on a series-parallel graph. Let denote the set of global minima of the Rosenthal potential function .

1. For any , there is a 1-lookahead outcome with .

2. All 1-lookahead outcomes of are global minima of the potential function and