Infinite-Duration All-Pay Bidding Games

05/12/2020
by   Guy Avni, et al.
0

A graph game is a two-player zero-sum game in which the players move a token throughout a graph to produce an infinite path, which determines the winner or payoff of the game. In "bidding games", in each turn, we hold an 'auction' (bidding) to determine which player moves the token. The players simultaneously submit bids and the higher bidder moves the token. Several different payment schemes have been considered. In "first-price" bidding, only the higher bidder pays his bid, while in "all-pay" bidding, both players pay their bids. Bidding games were largely studied with variants of first-price bidding. In this work, we study, for the first time, infinite-duration all-pay bidding games, and show that they exhibit the elegant mathematical properties of their first-price counterparts. This is in stark contrast with reachability games, which are known to be much more complicated under all-pay bidding than first-price bidding. Another orthogonal distinction between the bidding rules is in the recipient of the payments: in "Richman" bidding, the bids are paid to the other player, and in "poorman" bidding, the bids are paid to the 'bank'. We focus on strongly-connected games with "mean-payoff" and "parity" objectives. We completely solve all-pay Richman games: a simple argument shows that deterministic strategies cannot guarantee anything in this model, and it is technically much more challenging to find optimal probabilistic strategies that achieve the same expected guarantees in a game as can be obtained with deterministic strategies under first-price bidding. Under poorman all-pay bidding, in contrast to Richman bidding, deterministic strategies are useful and guarantee a payoff that is only slightly lower than the optimal payoff under first-price poorman bidding. Our proofs are constructive and based on new and significantly simpler constructions for first-price bidding.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

04/12/2018

Infinite-Duration Poorman-Bidding Games

In two-player games on graphs, the players move a token through a graph ...
05/09/2019

Bidding Mechanisms in Graph Games

In two-player games on graphs, the players move a token through a graph ...
10/20/2019

Leadership scenarios in prisoner's dilemma game

The prisoner's dilemma game is the most known contribution of game theor...
05/09/2019

Determinacy in Discrete-Bidding Infinite-Duration Games

In two-player games on graphs, the players move a token through a graph ...
11/19/2019

All-Pay Bidding Games on Graphs

In this paper we introduce and study all-pay bidding games, a class of ...
09/07/2020

One-Clock Priced Timed Games with Arbitrary Weights

Priced timed games are two-player zero-sum games played on priced timed ...
07/20/2016

Optimising Rule-Based Classification in Temporal Data

This study optimises manually derived rule-based expert system classific...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Graph games are two-player zero-sum games with deep connections to foundations of logic [25] as well as numerous practical applications, e.g., verification [15], reactive synthesis [23], and reasoning about multi-agent systems [2]. The game proceeds by placing a token on one of the vertices and allowing the players to move it throughout the graph to produce an infinite trace, which determines the winner or payoff of the game. Traditionally, the players alternate turns when moving the token. In bidding games [19, 18], however, the players have budgets, and in each turn, we hold an “auction” (bidding) to determine which player moves the token.

Four concrete bidding mechanisms have been previously defined. In all mechanisms, in each turn, both players simultaneously submit a bid that does not exceed their available budget, and the higher bidder moves the token. The mechanisms differ in their payment schemes, which we classify according to two orthogonal properties. First, in

first-price bidding only the higher bidder pays his bid, whereas in all-pay bidding, both players pay their bids. Second, in Richman bidding (named after David Richman) the payment(s) are paid to the other player whereas in poorman bidding the payments are paid to the “bank”, thus the money is lost. We refer to the mechanisms using abbreviations in . For example, FP-poor refers to first-price poorman and AP-Rich refers to all-pay Richman. The central quantity in bidding games is the budget ratio; for , when Player ’s budget is , then his budget ratio is .

Reachability FP-Rich and FP-poor games were studied in [19, 18]; each player has a target vertex, and the game ends once one of the targets is reached. It was shown that each vertex of the game has a threshold ratio, which is a necessary and sufficient initial budget for winning the game. Moreover, if a player has a sufficient budget for winning, he has a deterministic winning strategy. Only under FP-Rich bidding do reachability games have an intriguing equivalence with a class of games called random-turn games [22], in which in each turn, we toss a coin to determine which player moves the token (see more details in [18, 4]).

Reachability AP-poor games were only recently studied [7]. Technically, these games are significantly harder than first-price bidding: e.g., optimal strategies randomize over biddings, and already in very simple games, infinite support is required. Moreover, fundamental questions about this model are open.

Figure 1: The mean-payoff game with the weights in the vertices.
Figure 2: Max’s budget updates in four bidding outcomes under AP-Rich.

Infinite-duration bidding games were studied under FP-Rich [4] and FP-poor [5] bidding. For qualitative objectives, e.g., parity, bidding games reduce to reachability first-price bidding games by showing that one of the players wins a strongly-connected game with any positive initial ratio. Things get more interesting with mean-payoff objectives, which are quantitative (see Fig. 2 for an example): an infinite play has a payoff which is Player ’s reward and Player ’s cost, thus in these games the players are called Max and Min, respectively. The central question is identifying the optimal payoff Max can guarantee in a mean-payoff game with an initial budget ratio .111Technically, is optimal when Max can guarantee a payoff that is greater than , for every .

Mean-payoff first-price bidding exhibit intriguing equivalences with random-turn games. For a bias , we denote by

, the random-turn game in which in each turn, we toss a coin that selects Max to move with probability

and Min with probability . The game is a mean-payoff stochastic game [13], its mean-payoff value, denoted , is a well-known concept [24]. Under FP-Rich bidding, the initial budget ratio does not matter, and the optimal payoff Max can guarantee is . Thus, similar to reachability FP-Rich games, mean-payoff FP-Rich games are equivalent to fair random-turn games. Since such an equivalence is not known for reachability FP-poor games, it is surprising that mean-payoff FP-poor games are equivalent to random-turn games and the equivalence is in fact richer than for FP-Rich bidding: with an initial ratio of , the optimal payoff Max can guarantee under FP-poor bidding is . Thus, the initial ratio matters and coincides with the bias of the coin in the random-turn game. For example, in the game depicted in Fig. 2, with a ratio of , Max prefers FP-poor over FP-Rich, since with the first he can guarantee a payoff of and with the second, only .

We study, for the first time, infinite-duration all-pay bidding games. The starting point of this research is inspired by the results for FP-poor bidding: the moral of these results is that as we “go to the infinity”, bidding games become cleaner and exhibit a more elegant mathematical structure. We ask: Does this phenomenon also hold for all-pay bidding, where reachability games are highly complex? Would infinite-duration all-pay bidding games reveal a clean mathematical structure like their first-price counterparts? We answer both of these questions positively.

Before surveying our results, we note that in terms of applications, all-pay bidding is often better suited for modelling practical applications than first-price bidding. Applications arise from viewing the players’ budgets as resources with little or no inherent value, e.g., time or strength, and a strategy as a recipe to invest resources with the goal of maximizing the expected utility. In many settings, invested resources are lost, thus all-pay bidding is more appropriate than first-price bidding. Reachability AP-poor games [7] can be seen as a dynamic variant of Colonel Blotto games [9], which have been extensively studied. Applications of Colonel Blotto games, which carry over to all-pay bidding games, include political lobbying and campaigning, rent seeking [27], and modelling biological processes [12]. In fact, due to their dynamic nature, bidding games are a better model for these applications.

Another application of bidding games is reasoning about systems in which the scheduler accepts payment in exchange for priority. Blockchain technology is one such example. Simplifying the technology, a blockchain is a log of transactions issued by clients and maintained by miners, who accept transaction fees from clients in exchange for writing transactions to the blockchain. In Etherium, the blockchain consists of snippets of code (called smart contracts). Verification of Etherium programs is both challenging and important since bugs can cause loss of money (e.g., [11]). Bidding games, and specifically AP-poor games, can model Etherium programs: we associate players with clients and, as is standard in model checking, we associate the states of the program with the vertices of the graph. AP-poor bidding is the most appropriate bidding mechanism since in Etherium, the transaction fees are always paid to the miners.

We start by studying all-pay Richman biding and showing a complete picture for this bidding mechanism. We focus on games played on strongly-connected graphs. A simple argument shows that a deterministic Max strategy cannot guarantee anything under AP-Rich bidding. We turn to study probabilistic strategies, which is technically more challenging. We show that mean-payoff AP-Rich games are essentially equivalent to their first-price counterparts: in a game , with any positive initial ratio, Max’s optimal probabilistic strategy guarantees an expected payoff that equals the value of under FP-Rich, which in turn equals ; namely, the value of the fair random-turn game.

In parity AP-Rich games, we show that one of the players has a randomized strategy that guarantees winning with probability , and the winner depends only on the highest parity index in the game. Showing, again, an equivalence between parity AP-Rich and FP-Rich games. Our almost-sure winning strategy in a parity AP-Rich game is obtained by evoking carefully and repeatedly an optimal strategy in a mean-payoff AP-Rich game so that even though the guarantee in is on the expected payoff, we obtain almost-sure guarantees in .

Our solution of mean-payoff AP-Rich games is based on a new significantly-simpler construction of optimal strategies in mean-payoff FP-Rich games. The previous constructions of optimal FP-Rich strategies [4, 6] use the history of the play to determine the next bid; namely, the bid is roughly a function of the difference between the number of Max wins and loses during the play. It is technically not possible to use such a strategy in AP-Rich, as we illustrate in the example below. We devise new FP-Rich strategies that are history independent: the bid depends only the current budget and vertex, hence the name budget-based strategies, which are interesting in their own right.

We describe a simple Max strategy for the mean-payoff game that is depicted in Fig. 2, which achieves an expected payoff of under AP-Rich bidding; still not optimal, but better than any deterministic strategy can achieve. We start with the following observation. Suppose Max chooses a bid uniformly random from , for some . We assume Min wins ties. Thus, knowing Max’s strategy, Min chooses between deterministically bidding or . There are four possible outcomes (see Table 2). The “bad” outcomes for Max are and since Min wins without any budget penalty. The two other outcomes are “good” since they are similar to FP-Rich bidding: Max pays for winning and gains when losing. To obtain a probabilistic strategy for AP-Rich, we use an optimal strategy for FP-Rich as in [4], which achieves a payoff of by guaranteeing that Max wins as many biddings as Min (up to a constant). We input to the difference between Max wins and losses when restricting to good outcomes to obtain . Intuitively, we expect half the outcomes in a play to be good, out of these, guarantees that Max roughly wins half the biddings, for a total payoff of .

The construction above relies on a classification of outcomes to “good” and “bad”. To obtain an optimal strategy for AP-Rich, we would like decrease to a minimum the probability mass of the “bad” outcomes by bidding uniformly at random in . But then, it is not clear how to use to obtain . For example, when Min bids , both when Max bids and , he wins the bidding. However, Max is “luckier” in the first outcome since he pays less for winning. To overcome this issue, we devise a new optimal budget-based bidding strategy for FP-Rich. Intuitively, the current budget reflects precisely how “lucky” Max was in the previous biddings. We construct an optimal strategy under AP-Rich that bids uniformly at random from . ∎

Finally, we study mean-payoff all-pay poorman bidding. Poorman bidding games tend to be unpredictable and technically more challenging than Richman bidding. AP-poor is no exception. We show that, contrary to AP-Rich, deterministic strategies are useful under AP-poor bidding. Consider a game and suppose Max’s ratio is . We show that the optimal payoff Max can guarantee with a deterministic strategy in is . Not too far from the optimal payoff under FP-poor, which is . The result immediately implies that in a parity AP-poor game, one of the players wins with a ratio greater than . Here too, we first revisit mean-payoff FP-poor games and construct optimal budget-based strategies, which are significantly simpler than previous constructions. We leave open the problem of finding optimal probabilistic strategies for AP-poor bidding.

Further related work: All the results surveyed above highly depend on the fact that the players’ bids can be arbitrarily small. This is a problematic assumption for practical applications. To address this limitation discrete bidding games were studied in [14], where the budgets are given in “cents” and the minimal positive bid is one cent. Their motivation came from recreational play like bidding chess [8, 17]. AP-poor is not suited for discrete bidding since the budgets run out quickly. Discrete AP-Rich bidding has been studied in [21] (we encourage the reader to try playing AP-Rich tic-tac-toe online: http://tiny.cc/hqbgoz). While the issue of tie breaking does not play a key role in continuous bidding, it is important in discrete bidding [1]. Non-zero-sum FP-Rich games were studied in [20].

2 Preliminaries

A bidding game is a two-player game that is played on a directed graph , where is a finite set of vertices and is a set of directed edges. The neighbors of a vertex is the set of vertices . A path in is a finite or infinite sequence of vertices such that for every , we have . We denote by , the set of simple cycles in . We call a bidding game strongly-connected when the graph is strongly-connected. For , we use as short for when referring to the “other player”.

2.1 Bidding games, strategies, and plays.

The game proceeds as follows. We place a token on one of the vertices in the graph. In each turn, we hold a bidding to determine which player moves the token. Formally, a strategy is a recipe for how to play a game. It is a function that, given a finite history

of the game, prescribes to a player a probability distribution over

actions to take, where we define these two notions below. A strategy is deterministic when it always prescribes one action with probability .

Actions in bidding games are pairs of the form , where is a bid and is the vertex to move the token to upon winning the bidding. Suppose Player , for , bids in a bidding. The winner of the bidding is Player  if and otherwise it is Player . That is, we resolve ties in favor of Player . This is an arbitrary decision and our results are not affected by the tie-breaking mechanism that is used. A history in a bidding game is , where for , the token is placed on vertex at round , and , denotes Player ’s bid at round , for . An action is legal following if (1) is a neighbor of , thus a history gives rise to a path in the graph, which we denote by , and (2) the bid does not exceed the available budget as we define below.

For , we denote by the indices in which Player  wins and loses biddings, respectively. Suppose Player ’s initial budget is . The available budget of Player  following , denoted , depends on the bidding mechanism, and we consider four bidding mechanisms that differ in the payment scheme in each bidding:

  • First-price Richman: only the winner pays the loser, thus .

  • First-price poorman: only the winner pays the “bank”, thus .

  • All-pay Richman: both players pay each other, thus .

  • All-pay poorman: both players pay the “bank”, thus .

(Budget-based strategy). A strategy is called budget-based if it is independent of the history of the play and it is a function from the current vertex and budget to a vertex and a probability distribution on bids.

An initial vertex and two strategies and for the players give rise to a probability distribution over infinite plays, which we denote by , where we omit the initial vertex since it usually does not play a role in our results. When and are deterministic, there is a unique play that gets probability in , and we denote it by . It is defined inductively as follows. The initial vertex is . Let be a finite play that ends in , we define its continuation as follows. Let and . Then, if , we define and otherwise . Since we consider probabilistic strategies with continuous support, the definition requires us to define a probability space using a cylinder construction [3, Theorem 2.7.2], which is technical but standard and we do not present it here. When and are clear from the context, we omit them and simply write and instead of and expectation .

2.2 Objectives and values

The central quantity in bidding games is the ratio between the players’ budgets. (Budget ratio). For , let be Player ’s budget. Then, Player ’s ratio is . In Richman bidding, since the money only exchanges hands, the sum of budgets is constant and we normalize it to . In poorman bidding, we often normalize Player ’s budget to .

A qualitative objective is a set of infinite paths. The central question in qualitative bidding games concerns the necessary and sufficient initial ratio for guaranteeing that an objective is satisfied. The qualitative objectives that we consider are:

  • Reachability: Player  has a target vertex and an infinite play is winning iff it visits .

  • Parity: Each vertex is labeled with an index in . An infinite path is winning for Player 

    iff the parity of the maximal index that is visited is odd.

(Winning strategies). Consider a Player  strategy . We say that surely wins if it is deterministic and for every deterministic strategy , we have . We say that almost-surely wins if for every strategy , we have .

The quantitative objective we focus on is mean-payoff. Each play in a mean-payoff game has a payoff, which is Player ’s reward and Player ’s cost, thus in mean-payoff games, we refer to Player  as Max and to Player  as Min. The central question in mean-payoff bidding games concerns the optimal payoff a player can guarantee with an initial ratio, called the mean-payoff value. Technically, a mean-payoff game is played on a weighted directed graph , where . Consider an infinite path . For , the prefix of length of is . The energy of is . We define the payoff of an infinite and finite paths. For an infinite path and , we define and . Note that the definition gives Min an advantage. For , let and respectively denote the set of deterministic and probabilistic strategies for Player  with an initial ratio of .

(Mean-payoff value). Consider a mean-payoff game and a ratio .

  • The expected-value of w.r.t. , denoted , is if for every and no matter where the game starts, Max has a probabilistic strategy s.t. for every , we have , and dually, there is s.t. for every , we have .

  • The sure-value of w.r.t. , denoted , is if Max can deterministically guarantee a payoff of : for every and no matter where the game starts, there is s.t. for every , we have . And, Max cannot do better: for every Max strategy , there is that guarantees .

We sometimes write , for to highlight the bidding mechanism that is used. We note that in order to show that guarantees an expected payoff of , it suffices to show that it guarantees an expected payoff of against every deterministic strategy .

2.3 Random-turn games, strengths, and potentials

In this section, we describe the tool that, intuitively, allows us to extend a solution for the game (Fig. 2) to a general strongly-connected game. In such games, we need a measure of how “important” it is to choose the successor of a vertex, which we call the strength. The higher the strength of , the higher a player should bid while the token is in . To define strengths, we need several definitions.

A random-turn game [22] parameterized by is played on a graph as a bidding game, only that instead of bidding for moving, we throw a (biased) coin to determine which player moves the token: Player  is chosen with probability and Player  with probability . Formally, a random-turn game is a stochastic game [13]. The objective in matches that of , and we focus on strongly-connected mean-payoff random-turn games. The mean-payoff value of , denote , is defined as the expected payoff that a player can guarantee. It is well-known that it is achievable using positional strategies [24]. Such a strategy takes as input the vertex the token is placed on, and returns which vertex to move the token to upon being selected by the coin toss. Let and respectively denote optimal positional strategies for Max and Min. For , we denote and . The concept of potentials was originally defined in the context of the strategy iteration algorithm [16]. We denote the potential of a vertex by and the strength of by , and we define them as solutions to the following equations.

There are optimal strategies for which , for every neighbor of , which can be found, for example, using the strategy iteration algorithm. Note that , for every . We denote the maximal strength by and we assume otherwise the game is trivial as all weights are equal.

Consider a finite path in . We intuitively think of as a play, where for every , the bid of Max in is and he moves to upon winning. Thus, when , we think of Max as investing and when , we think of Min winning the bid thus Max gains . We denote by and the sum of investments and gains, respectively. Note that and are defined w.r.t and will be clear from the context. Recall that the energy of is . The following lemma connects the energy, potentials, and strengths.

[6] Consider a strongly-connected game , and . For every finite path in , we have . In particular, when for , there is a constant such that .

The proof of the following corollary can be found in App. A.

Consider a strongly-connected game , let be an infinite play, let , and . If for every finite prefix of , then .

3 All-Pay Richman Bidding Games

In this section we completely solve infinite-duration all-pay Richman bidding games. We focus on strongly-connected games. We start with the following negative result that shows that deterministic strategies are useless in all-pay Richman bidding.

Let be a strongly-connected game. For any initial ratio and a deterministic Player  strategy, Player  has a strategy that wins all but a constant number of biddings. Specifically,

  1. When is a mean-payoff game, the sure-value of is the lowest possible; namely, ;

  2. When is a parity game that has a cycle in with an even maximal parity index, then no deterministic Player  strategy can guarantee winning with a positive probability.

Proof.

Let be Player ’s initial budget. Consider a deterministic Player  strategy that, for , bids in the -th bidding. Player  bids when , and other he bids . Suppose Player ’s initial budget is . Since whenever Player  loses a bidding, Player  bids at least , and the sum of Player ’s payments to Player  in a finite play is less than , Player  can win at most times. ∎

3.1 Revisiting mean-payoff first-price Richman games

Our construction of optimal probabilistic strategies in mean-payoff AP-Rich games is based on a new significantly-simpler construction of budget-based strategies for mean-payoff FP-Rich games. Our constructions throughout the paper use the following definition: (Shift function). The shift function is defined as . The proof of the following lemma can be found in App. C. The shift function has the following properties:

  • For every , there exists such that .

  • For every , we have .

Previous constructions of optimal strategies for mean-payoff FP-Rich games can be found in [4, 6]. Below we revisit mean-payoff FP-Rich games and devise optimal budget-based strategies. For ease of presentation, we illustrate the construction on the simple game , and it can easily be extended to general SCCs, as we note in the proof.

Consider the game , depicted in Fig. 2. Under first-price Richman bidding, for every initial ratio and , Max has a deterministic budget-based strategy that guarantees a payoff of at least , thus .

Proof.

Let be Max’s initial budget. It is convenient to set the weights in the game to and , where depends on , and setting Max’s goal to keep the energy bounded from below by a constant. Then, as the length of the play tends to infinity, the ratio of Max’s wins tends to , thus the payoff in the original game is . Let such that (see Lem. 3.1). Max maintains the invariant that when the energy is , his budget exceeds . We choose an initial such that the invariant holds for , which is possible since . Max keeps the energy above .

Max plays as follows. When Max’s budget is , he bids . Note that the strategy is budget based since the bid depends only on the current budget.222For general SCCs, at a vertex , we bid , see more details in Lem. 3.2. To conclude the proof, we show that the invariant is maintained and that it guarantees that . We distinguish between the two outcomes of a bidding. If Max loses, the energy decreases to . Moreover, Min overbids Max, thus Max’s new budget is at least . On the other hand, if Max wins, the energy increases to and his new budget is at least . By plugging in and in Lemma 3.1, we obtain , thus , as required. To conclude the proof, we have , since otherwise, by the invariant, Max’s budget would be , which is impossible since the budgets sum up to in Richman bidding. ∎

3.2 Mean-payoff all-pay Richman games with probabilistic strategies

This section consists of our main technical contribution on all-pay Richman bidding. We find optimal probabilistic strategies and show that they do not depend on the initial ratio, thereby showing equivalence between mean-payoff all-pay and first-price Richman games.

Let be a strongly-connected mean-payoff all-pay Richman bidding game. For every initial ratio of Max, for every , Max has a probabilistic budget-based strategy in that guarantees an expected payoff of at least .

Proof.

Let , and let . We find vertex strengths using as in Section 2.3. Let s.t. (see Lemma 3.1). We define Max’s strategy as follows.

  • When the token is on vertex with strength , and Max’s budget is , Max bids .

  • Upon winning, Max moves the token to .

We claim that no matter which deterministic strategy Min chooses, we have . We assume Wlog that Min bids , since he has the tie-breaking advantage and does not profit from bidding higher.

Proofs in first-price bidding show an invariant between changes in energy and changes in budget (see Prop. 3.1). For all-pay bidding, however, such an invariant is not possible. Our invariant uses a new component, which we refer to as “luck”. Intuitively, a Max bid of is “unlucky” when is much larger than or when is slightly smaller than . In the first case, Max pays a lot for winning and in the second, Min pays little for winning. The top-left to bottom-right diagonal in Tab. 2 take these scenarios to the extreme. Dually, when is just above or way below , Max is “lucky”. See the other diagonal in the table. The key idea of the proof is that on average, the unlucky and lucky cases cancel out.

To formalize the notion of luck, we define a quantity as follows. Initially, we set . We define the change in luck following a bidding

(3.1)

We formalize the intuition that lucky and unlucky events cancel out with the following claim, proved in App. D.

Claim: Suppose that following a finite play, Max bids , then for every Min bid , we have

(3.2)

For any finite play , recall that and are the sum of the strengths of the vertices where Max wins, respectively loses. We prove in App. E that the budget of Max after the play satisfies the following invariant, by induction on the length of the play and using Bernoulli’s inequality.

Claim: For every finite play coherent with the strategy of Max, we have

(3.3)

Since the sum of budgets is , we have . Comparing the exponents, we obtain . Since at each turn and for each , we conclude that for any deterministic strategy of Min we have where the expectation is taken over the probability space of all plays defined by and . Combining with Lemma 2.3, by plugging and , we finally obtain

(3.4)

Dividing by , we obtain , and we are done. ∎

Since is continuous in [10, 26], it follows from Lemma 3.2 that , for every . Note that since the definition of payoff favors Min, Min can follow Max’s strategy above to show that Max cannot achieve a payoff of . We thus have the following.

Let be a strongly-connected mean-payoff all-pay Richman bidding game. The expected mean-payoff value in equals the surely mean-payoff value in under first-price bidding, which in turn equals the mean-payoff value of the random-turn game in which the player who moves is chosen uniformly at random, thus for every budget ratio , we have .

3.3 Qualitative all-pay Richman games

In this section we consider strongly-connected parity AP-Rich games. Recall that under FP-Rich, one of the players wins with any initial budget, and the winner depends on the parity of the highest parity index. Analogously, we show that in parity AP-Rich games, one of the players wins with probability with any positive initial budget.

Let be a parity game in which the maximal parity index is . We construct a mean-payoff game by setting the weight of a vertex to be if the parity of is , and otherwise . The key property of this weight function is that any path with must visit a vertex with index infinitely many times, and thus satisfies the parity objective. The proof of the following lemma can be found in App. F.

Let be a strongly-connected mean-payoff game with non-negative weights and at least one strictly positive weight. Then, for every , we have .

The challenge in using an optimal strategy for in is that the guarantees in are on the expected payoff, and they do not immediately imply almost-sure guarantees in . We manage to recover these guarantees by carefully and repeatedly invoking to obtain the following result.

Let be a strongly-connected parity all-pay Richman bidding game in which the highest parity is . Then, the player corresponding to the parity of has a randomized almost-sure winning strategy, no matter where the game starts and with which positive initial ratio.

Proof.

We assume WLog that the highest parity index is odd and denote by , the set of vertices with parity index . We construct as in the above. Since is strongly-connected, for each by Thm. 3.2, we have . Moreover, the proof of Lemma 3.2 shows that there exists a strategy which guarantees a payoff of at least independent of the initial vertex and the initial budget. By Lemma 3.3, we have for some .

Next, we define a function that takes an initial ratio , and returns an index such that for every , for every strategy of the opponent we have . It follows from eq. (3.4), that , where is a constant depending on . We define hence, as required, for every we have

The proof of the following claim can be found in App. G.

Claim: If Player  follows the strategy with an initial budget , then the probability of visiting a vertex in during the first steps is at least .

We are now ready to define the Player  strategy that almost-surely guarantees the parity objective. Let be the initial budget of Player . We partition the budget into portions , and the time over which the game evolves into time periods , where each is of length . Then for each , the strategy uses portion of the initial budget to follow during the time period .

We claim that guarantees almost-surely visiting infinitely often. We prove this by contradiction and suppose that there exists a strategy for the opponent such that is visited only finitely many times with positive probability (over the probability space defined by and ). This implies that

where the last inequality follows by the union bound. Hence, there exists such that . This event is equivalent to not being visited in time periods , which in turn implies that for every the probability of not being visited in between time and is also positive. But for a fixed this probability can be factorized as

(3.5)

A key point in the proof of the following claim is that the construction of is budget-based. As the budget used by the strategy at the beginning of time period is , by expanding each of the factors in eq. 3.5 as a sum over the initial vertex of the time period, one can deduce the claim.

Claim: For each we have

  • , and

  • . ∎

4 All-Pay Poorman Bidding Games

In this section, we construct optimal deterministic strategies in all-pay poorman games. We find it surprising that, contrary to all-pay Richman bidding games (Theorem 3), deterministic strategies are useful in all-pay poorman bidding games. Throughout this section it is technically convenient to keep Min’s budget normalized to , thus for a budget of Max, we consider the ratio .

4.1 Revisiting mean-payoff first-price poorman games

The value of mean-payoff first-price poorman games was first identified in [5].

[[5]] Let be a strongly-connected mean-payoff all-pay poorman bidding game. For every initial ratio of Max, we have .

We revisit this result and provide an alternative proof by constructing new and significantly simpler optimal budget-based bidding strategies. We then base our solution to all-pay bidding on the budget-based strategy.

Let be a strongly-connected mean-payoff all-pay poorman bidding game. For every initial ratio of Max, for every , Max has a deterministic budget-based strategy that guarantees a payoff of .

Proof.

Let be Max’s initial budget. As usual, we keep Min’s budget normalized to . Let , and let . We design a deterministic strategy for Max that maintains his budget above . Then, the value of the updated budget of Max after a bidding where Max bids and Min bids is as follows: If Max wins the bidding (), we have . The key new insight is that when Max loses the bidding (), we have

(4.1)

Intuitively, the property states that every cent is times more valuable to Min than it is to Max. For example, if Max’s budget is and Min’s budget is , then paying is twice as painful for Min as it is for Max. Roughly, on average, this means that Max wins times more biddings than Min, thus he guarantees a payoff close to .

We now proceed to define formally a budget-based bidding strategy for Max that guarantees a payoff of at least , where . We pick satisfying (see Lemma 3.1). We find vertex strengths using as in Section 2.3. Let . The strategy is defined as follows.

  • When the token is placed on a vertex with strength and Max’s budget is , Max bids .

  • Upon winning, Max moves the token to .

We first show that Max’s bidding strategy is legal, by showing that we always have . Indeed, initially, we have , and then whenever Max loses a bidding his budget increases, and when Max wins a bidding his updated budget is , which is still greater than since .

Next, for any finite play , let . Recall that and denote the sum of the strengths of the vertices of where Max wins, respectively loses. We prove in App. H that the budget of Max after the play satisfies the following invariant, using induction on the length of and Bernoulli’s inequality.

Claim: For every finite play coherent with the strategy of Max, we have

(4.2)

As a consequence, we show the existence of a bound such that for every finite play coherent with .

Claim: There exists such that for every finite play coherent with , we have

(4.3)

The proof of the claim, found in App I, can be summarised as follows. Since the left-hand side of the equation is equal to , proving the claim is equivalent to proving a lower bound for . To do so, we show that cannot get too low, as past some threshold Equation (4.2) guarantees that the budget of Max is so high that his next bid according to the strategy will be above (the whole budget of Min). Thus Max is guaranteed to win the next bidding, which results in going back up.

Combining Claim 4.3 with Corollary 2.3 (plugging and ), we finally obtain that any infinite play coherent with the strategy has a mean-payoff greater than . ∎

4.2 Infinite-duration all-pay poorman-bidding games

This section consists of our main technical contribution for all-pay poorman bidding. We first solve mean-payoff games, and then use our solution to solve parity games. Recall that, the value of a strongly-connected mean-payoff game w.r.t. an initial ratio under FP-poor is . The following theorem shows that the sure-value under all-pay poorman bidding is not far from the sure-value under first-price poorman bidding.

Let be a strongly-connected mean-payoff all-pay poorman bidding game. For every initial ratio of Max, we have .

Since is continuous in [10, 26], we demonstrate the theorem by proving the following lemma.

Let be a strongly-connected mean-payoff all-pay poorman bidding game. For every initial ratio of Max, for every ,

  1. Max can deterministically guarantee a payoff of ;

  2. For any fixed deterministic strategy of Max, Min can guarantee a payoff of .

Proof.

In App. J, we show that the strategy that we constructed in Prop. 4.1 for FP-poor guarantees a payoff of under AP-poor. The analysis needs to be adjusted to AP-poor. The main difference is that the change in Max’s budget following losing a bidding to a bid of Min is now

For Item ii, consider a deterministic strategy of Max. We describe a Min strategy as follows. Let , let , and we choose satisfying . For each vertex with strength and a Max budget , Min fixes a threshold . When the token is placed on , Min computes the bid that Max will bid according to . If , Min bids , wins the bidding and moves to . If , Min judges that the budget required to win the bidding is not worth it, and stays out by bidding . In App. K, we show that guarantees a payoff of . Intuitively, technically, the threshold is where the invariant on Max’s budget is (close to) an equality no matter the bid of Min. Thus, if Max bids higher than , he is paying “too much” for a win and eventually his budget will run out. ∎

In App. L, we show that a construction of a mean-payoff game from a parity game as in Sec. 3.3 together with Lem. 4.2 provides the following.

Consider a strongly-connected parity game in which the highest priority is odd. For every initial ratio , Player  has a surely winning strategy in .

5 Discussion

We study, for the first time, infinite-duration all-pay bidding games. We show a complete picture for all-pay Richman bidding: deterministic strategies cannot guarantee anything, and with probabilistic strategies, AP-Rich coincides with FP-Rich both for mean-payoff and parity objectives. For all-pay poorman bidding, we show that, surprisingly, deterministic strategies can guarantee a payoff not too far from the optimal payoff under FP-poor bidding.

We leave open the problem of classifying the expected value in a mean-payoff AP-poor bidding game . It is tempting to conjecture that as in AP-Rich bidding, the expected value coincides with the sure-value under first-price bidding.However, this might not be true due to the following difference between the bidding rules. When both players bid (see the bottom-right cell in Table 2), the situation under AP-Rich bidding is worse than under AP-poor bidding. Indeed, with AP-Rich, Max’s updated budget is , whereas with AP-poor, his budget is , and we have when . This slight difference might lead to .

Taxman bidding spans the spectrum between Richman and poorman bidding: for a constant , portion is paid to the other player player and portion is paid to the bank. Mean-payoff first-price taxman bidding games were studied in [6] with the motivation to better understand the differences between FP-Rich and FP-poor bidding. The same motivation applies to all-pay bidding, and we find it interesting to study all-pay taxman bidding. Unlike first-price bidding, with all-pay bidding classifying both the sure- and expected-value for all-pay taxman bidding are interesting problems.

In terms of computational complexity, since solving stochastic games is in NP and coNP, the equivalences between strongly-connected mean-payoff bidding games and random-turn games implies the same upper bound on the problem of finding the sure or expected value of the corresponding bidding game. Finding a lower bound, e.g., that bidding games are harder than general stochastic games, is an open problem also for first-price bidding.

References

  • [1] M. Aghajohari, G. Avni, and T. A. Henzinger. Determinacy in discrete-bidding infinite-duration games. In Proc. 30th CONCUR, volume 140 of LIPIcs, pages 20:1–20:17, 2019.
  • [2] R. Alur, T. A. Henzinger, and O. Kupferman. Alternating-time temporal logic. J. ACM, 49(5):672–713, 2002.
  • [3] R.B. Ash and C. Doléans-Dade. Probability and Measure Theory. Harcourt/Academic Press, 2000.
  • [4] G. Avni, T. A. Henzinger, and V. Chonev. Infinite-duration bidding games. J. ACM, 66(4):31:1–31:29, 2019.
  • [5] G. Avni, T. A. Henzinger, and R. Ibsen-Jensen. Infinite-duration poorman-bidding games. In Proc. 14th WINE, volume 11316 of LNCS, pages 21–36. Springer, 2018.
  • [6] G. Avni, T. A. Henzinger, and Đ. Žikelić. Bidding mechanisms in graph games. In In Proc. 44th MFCS, volume 138 of LIPIcs, pages 11:1–11:13, 2019.
  • [7] G. Avni, R. Ibsen-Jensen, and J. Tkadlec. All-pay bidding games on graphs. Proc. 34th AAAI, 2020.
  • [8] J. Bhatt and S. Payne. Bidding chess. Math. Intelligencer, 31:37–39, 2009.
  • [9] E. Borel. La théorie du jeu les équations intégrales á noyau symétrique. Comptes Rendus de l’Académie, 173(1304–1308):58, 1921.
  • [10] K. Chatterjee. Robustness of structurally equivalent concurrent parity games. In Proc. 15th FoSSaCS, pages 270–285, 2012.
  • [11] K. Chatterjee, A. K. Goharshady, and Y. Velner. Quantitative analysis of smart contracts. In Proc. 27th ESOP, pages 739–767, 2018.
  • [12] Krishnendu Chatterjee, Johannes G. Reiter, and Martin A. Nowak. Evolutionary dynamics of biological auctions. Theoretical Population Biology, 81(1):69 – 80, 2012.
  • [13] A. Condon. The complexity of stochastic games. Inf. Comput., 96(2):203–224, 1992.
  • [14] M. Develin and S. Payne. Discrete bidding games. The Electronic Journal of Combinatorics, 17(1):R85, 2010.
  • [15] A. E. Emerson, C. S. Jutla, and P. A. Sistla. On model-checking for fragments of -calculus. In Proc. 5th CAV, pages 385–396, 1993.
  • [16] A. R. Howard. Dynamic Programming and Markov Processes. MIT Press, 1960.
  • [17] U. Larsson and J. Wästlund. Endgames in bidding chess. Games of No Chance 5, 70, 2018.
  • [18] A. J. Lazarus, D. E. Loeb, J. G. Propp, W. R. Stromquist, and D. H. Ullman. Combinatorial games under auction play. Games and Economic Behavior, 27(2):229–264, 1999.
  • [19] A. J. Lazarus, D. E. Loeb, J. G. Propp, and D. Ullman. Richman games. Games of No Chance, 29:439–449, 1996.
  • [20] R. Meir, G. Kalai, and M. Tennenholtz. Bidding games and efficient allocations. Games and Economic Behavior, 2018. doi:https://doi.org/10.1016/j.geb.2018.08.005.
  • [21] M. Menz, J. Wang, and J. Xie. Discrete all-pay bidding games. CoRR, abs/1504.02799, 2015.
  • [22] Y. Peres, O. Schramm, S. Sheffield, and D. B. Wilson. Tug-of-war and the infinity laplacian. J. Amer. Math. Soc., 22:167–210, 2009.
  • [23] A. Pnueli and R. Rosner. On the synthesis of a reactive module. In Proc. 16th POPL, pages 179–190, 1989.
  • [24] M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York, NY, USA, 2005.
  • [25] M.O. Rabin. Decidability of second order theories and automata on infinite trees. Transaction of the AMS, 141:1–35, 1969.
  • [26] E. Solan. Continuity of the value of competitive markov decision processes. Journal of Theoretical Probability, 16:831–845, 2003.
  • [27] G. Tullock. Toward a Theory of the Rent Seeking Society, chapter Efficient rent seeking, pages 97–112. College Station: Texas A&M Press, 1980.

Appendix A Proof of Corollary 2.3

Given an infinite play and , let us denote by the prefix of of length . Suppose that for every . Then applying Lemma 2.3 yields

This concludes the proof, as both fractions tend towards as approaches .

Appendix B Bernoulli’s inequality

Bernoulli inequality is stated as follows.

(B.1)

Appendix C Properties of the shift function

The shift function is surjective as

  1. (l’Hôpital rule);

  2. ;

  3. is continuous as its denominator is strictly positive over the domain, and is continuous.

As a consequence, for every , there exists such that . The second item is a direct consequence of the definition of the shift function.

Appendix D Proof of Equation 3.2

Let denote