Game of Coins

1 Introduction

Cryptocurrencies are an arms race. Hundreds of digital coins have crept into the worldwide market in the last decade MarketState , including more than a dozen with over a billion dollar Market Cap, e.g., buterin2014next ; BitcoinCash ; Litecoin ; Cardano ; Neo . The vast majority of cryptocurrencies are based on the notion of proof of work (PoW) Bitcoin . As a result, the major strategic players in the context of cryptocurrencies are miners who devote their power to solving computational puzzles to find PoWs Bitcoin ; buterin2014next .

The miners for a particular coin usually gain rewards that are proportional to the power they invest in the coin out of the total invested power (in the coin) by all miners. Each coin, therefore, can be viewed as having some weight that reflects the reward it divides among its miners. In practice, a coin’s weight (or reward) depends on its transaction rate, transaction fees, and its fiat exchange rate.

While the above description is not complete, it does capture the fundamental decision faced by the miner: where should I mine? One indication for reward-based coin switching can be found online in websites like www.whattomine.com whattomine , where miners enter their mining parameters (technology, power, cost, et cetra) and get a list of coins they can mine for, ordered by their profitability. Another interesting example happened on November 12 (2017) bitinfocharts , when a dramatic change in the Bitcoin to Bitcoin Cash BitcoinCash (a spin-off from Bitcoin) exchange rate led to a major inrush of miners from Bitcoin to Bitcoin Cash (see Figure 1).

(a) Bitcoin and Bitcoin Cash exchange rates over time.

All in all, the structure of the cryptocurrency market suggests that we face here a game among miners, where each miner wishes to mine coins of heavy weights while avoiding competition with other miners. In this paper we introduce for the first time the study of the cryptocurrency market as a game, consisting of a set of strategic players (miners) with possibly different mining powers and a set of coins with possibly different rewards (weights). The miners are free to choose to mine for any coin from the set, and we consider general better-response learning of the miners. That is, whenever any miner may benefit from deviating (i.e., changing the coin it mines for), some miner will take a step that improves his payoff; we allow an arbitrary sequence of such individual improvement steps (sometimes called improving path MondererShapley96 ). In our first major result we prove that any such better response learning converges to a (pure) equilibrium regardless of miner powers and coin rewards! This result is obtained by showing an ordinal potential, which according to MondererShapley96 , implies that arbitrary better response learning converges to equilibrium.

Having at hand the above fundamental result, we move to a discussion of strategic manipulation Nisan:2007:AGT:1296179 . While many efforts have been invested in the study of crypto-related manipulations SelfishMining ; OptimalSelfishMining ; StubbornMining ; MinersDilemma , we introduce for the first time the manipulation of the miners’ learning and optimization process. Given that a shift in the weight of a coin may influence miner behavior whattomine , in the cryptocurrency setting, it is quite possible for an interested party to affect this weight, either by creating additional transactions with high fees (sometimes called whale transactions liao2017incentivizing ) or by manipulating the coin exchange rate gandal2018price ; priceMuniplautionm ; whalesMuniplaution ; BitcoinMuniplaution . This way, a miner (or another interested party) can attempt to change the system equilibrium to a better one for them. We show that under broad circumstances, for every equilibrium of such a game, there exists a miner and another equilibrium in which the miner’s payoff is higher. The question is therefore: can one design rewards (i.e., temporarily increase coin weights) in a way that will lead the system from a given equilibrium to a desired one, so that the system will remain in the desired equilibrium after reverting to the original weights? Note that such reward design allows the manipulator to pay a finite cost while gaining an advantage indefinitely.

The above reward design problem is challenging since miners might take any better response step, and may make their moves in any order. Given the (modified) weights, we can use our previous major result to claim that any better response learning will converge to an equilibrium. Notice that the latter may not be the desired one, but now we can modify the rewards again. In the second major result of this paper we show that such desired reward design for learning agents is feasible! Namely, we provide a (multi-step) algorithm for assigning rewards in equilibrium states that moves learning agents from any initial equilibrium to a desired one.

In summary, our contributions are as follows:

We formalize strategic mining in multi-cryptocurrency markets as a game (Section 2).
We prove that any better-response learning in such games, starting from an arbitrary configuration, converges to equilibrium (Section 3).
We show that, in many cases, for every equilibrium there is a miner and another equilibrium in which the miner’s payoff is higher (Section 4).
We offer a reward design scheme that moves the system configuration from any initial equilibrium to a desired one for any better-response learning of the miners (Section 5).

For space limitations, the proofs of some of the claims we state here are deferred to Appendices C - F.

1.1 Related work

Results on better response learning convergence to pure equilibrium are rare and are typically restricted to games with exact potential NIPS2017_7216 ; MondererShapley96 , which coincide with congestion games. We show that our game does not have an exact potential (Section 3), and in fact our game belongs to the larger class of ID congestion games, where the payoff of a player depends on the player and the identity of other players who choose a similar resource, rather than on their number only. While there exist extensions of congestion games in which better-response learning converges to equilibrium (e.g., a restricted form of player-specific congestion games Milchtaich96 , which does not include our game), such results are extremely rare in the context of ID congestion games.

Unlike works on learning in games that emphasize adapting specific machine learning algorithms to minimize regret

NIPS2017_7216 ; PalaiopanosPP17 ; NIPS2015_5763 ; learninggamesbook , we assume minimal rationality on behalf of the players, i.e., that they follow an arbitrary better response step improving their individual payoffs.

Our work also expands literature on reward design NIPS2010_4146 ; NIPS2017_7253 ; SorgSL10

, and to the best of our knowledge, is the first to introduce reward design for learning agents in a multi-agent setting. While seminal works in reward design assign/modify state rewards in a reinforcement learning context

DBLP:books/lib/SuttonB98 , we design rewards for equilibrium states for any better response learning.

Though several previous works presented game theoretical analyses for cryptocurrencies liao2017incentivizing ; SelfishMining ; OptimalSelfishMining ; StubbornMining ; carlsten2016instability ; MiningGames ; Johnson14 ; MinersDilemma ; Schrijvers16 , the vast majority of them deal (in one way or another) with miners’ incentives to follow the coins’ mining protocols. Our work is the first to extend the study to a multi-coin setting and establish fundamental game theoretical results therein.

2 Model

A system in our model is a tuple $⟨ Π, C ⟩$ , where $Π$ is a finite set of $n$ miners (players) and $C$ is a finite set of coins (resources). A miner $p \in Π$ has mining power $m_{p} \in R_{+}$ , which it can invest in one of the coins , i.e., the set of possible actions of $p$ is $C$ . We denote the set of configurations of a system $Q = ⟨ Π, C ⟩$ as $S_{Q} ≜ C^{n}$ and denote by $s . p$ the action of player $p \in Π$ in configuration $s \in S_{Q}$ . When clear from the context, we omit the subscript indicating the system and simply write $S$ . Given $s \in S$ and $c \in C$ , we denote by $P_{c} (s) \subseteq Π$ the set of miners who mine for $c$ in $s$ , i.e., $P_{c} (s) ≜ {p \in Π ∣ s . p = c}$ , and by $M_{c} (s)$ their total mining power, i.e., $M_{c} (s) ≜ Σ_{p \in P_{c} (s)} m_{p}$ . For $s \in S_{Q}, p \in Π, c \in C$ we denote by $(s_{- p}, c)$ the configuration that is identical to $s$ except that $s . p$ is replaced by $c$ .

A reward function $F : C \to R_{+}$ maps coins to rewards. A game $G_{Π, C, F}$ consists of a system $⟨ Π, C ⟩$ and a reward function $F$ . Every coin in a game $G_{Π, C, F}$ divides its reward among all the players that mine for it, and the miners’ payoffs are defined as follows: For $s \in S$ , the revenue per unit (RPU) of coin $c$ in $s$ is $R P U_{c} (G_{Π, C, F}) (s) ≜ \frac{F (c)}{M_{c} (s)}$ . When clear from the context, we omit the the parameter indicating the game. The payoff function of a miner $p \in Π$ is $u_{p} (s) ≜ m_{p} \frac{F (s . p)}{M_{s . p} (s)} = m_{p} \cdot R P U_{s . p} (s)$ .

Given a game $G_{Π, C, F}$ , a configuration $s \in S$ , a miner $p \in Π$ , and a coin $c \in C$ , we say that $p$ moves from $s . p$ to $c$ in $s$ if it changes its action from $s . p$ to $c$ . A move from $s . p$ to $c$ is a better response step for $p$ if $u_{p} (s) < u_{p} ((s_{- p}, c))$ . We say that a miner $p \in Π$ is stable in a configuration $s$ in game $G_{Π, C, F}$ if $p$ has no better response steps in $s$ . A configuration $s$ is stable or a (pure) equilibrium if every miner $p \in Π$ is stable in $s$ . A better response learning from $s$ in $G_{Π, C, F}$ is a sequence of configurations resulting from a sequence of better response steps starting from $s$ , which is either infinite or ends with a stable configuration. In case it is finite, we say that it converges to its final configuration.

A function $f : S \to R$ is an ordinal potential for a game $G_{Π, C, F}$ if for any two configurations $s, s^{'} \in S$ s.t. some better response step of a miner $p \in Π$ leads from $s$ to $s^{'}$ , it holds that $f (s) < f (s^{'})$ . If, in addition, $f (s^{'}) - f (s) = u_{p} (s^{'}) - u_{p} (s)$ , then $f$ is an exact potential. By MondererShapley96 , if a game $G_{Π, C, F}$ has an ordinal potential, then every better response learning converges.

3 Better response learning convergance

In this section we prove that although a game $G_{Π, C, F}$ has no exact potential, every better-response learning of the miners in game $G_{Π, C, F}$ converges to a stable configuration (pure equilibrium) regardless of the sets $Π$ and $C$ and the reward function $F$ . To gain intuition, the reader is referred to our Appendix A and B, where we show how to construct a particular equilibrium in a game $G_{Π, C, F}$ for any $Π, C$ and $F$ , and give a simple ordinal potential function for the symmetric case in which $F$ is a constant function i.e., $\forall c, c^{'} \in C$ , $F (c) = F (c^{'})$ , respectively.

No exact potential.

We start by showing that our game does not have an exact potential.

Proposition 1.

The game $G_{Π, C, F}$ does not always have an exact potential.

Proof.

Let $G_{Π, C, F}$ be a game where $Π = {p_{1}, p_{2}}$ , $m_{p_{1}} = 2, m_{p_{2}} = 1$ , $C = {c_{1}, c_{2}}$ , and $F (c_{1}) = F (c_{2}) = 1$ . Assume by way of contradiction that $G_{Π, C, F}$ has an exact potential function $H$ , and consider the following four configurations:

$s^{1} = ⟨ c_{1}, c_{1} ⟩$ . Payoffs: $u_{p_{1}} (s^{1}) = \frac{F (c_{1}) \cdot m_{p_{1}}}{m_{p_{1}} + m_{p_{2}}} = 2 / 3$ , $u_{p_{2}} (s^{1}) = \frac{F (c_{1}) \cdot m_{p_{2}}}{m_{p_{1}} + m_{p_{2}}} = 1 / 3$ .
$s^{2} = ⟨ c_{1}, c_{2} ⟩$ . Payoffs: $u_{p_{1}} (s^{2}) = \frac{F (c_{1}) \cdot m_{p_{1}}}{m_{p_{1}}} = 1$ , $u_{p_{2}} (s^{2}) = \frac{F (c_{2}) \cdot m_{p_{2}}}{m_{p_{2}}} = 1$ .
$s^{3} = ⟨ c_{2}, c_{2} ⟩$ . Payoffs: $u_{p_{1}} (s^{3}) = \frac{F (c_{2}) \cdot m_{p_{1}}}{m_{p_{1}} + m_{p_{2}}} = 2 / 3$ , $u_{p_{2}} (s^{3}) = \frac{F (c_{2}) \cdot m_{p_{2}}}{m_{p_{1}} + m_{p_{2}}} = 1 / 3$ .
$s^{4} = ⟨ c_{2}, c_{1} ⟩$ . Payoffs: $u_{p_{1}} (s^{4}) = \frac{F (c_{2}) \cdot m_{p_{1}}}{m_{p_{1}}} = 1$ , $u_{p_{2}} (s^{4}) = \frac{F (c_{1}) \cdot m_{p_{2}}}{m_{p_{2}}} = 1$ .

Note that $H (s_{2}) - H (s_{1}) + H (s_{3}) - H (s_{2}) + H (s_{4}) - H (s_{3}) + H (s_{1}) - H (s_{4}) = 0$ . However, by definition of exact potential, we get that $(H (s_{2}) - H (s_{1})) + (H (s_{3}) - H (s_{2})) + (H (s_{4}) - H (s_{3})) + (H (s_{1}) - H (s_{4})) = (2 / 3) + (- 1 / 3) + (2 / 3) + (- 1 / 3) = 2 / 3 \neq 0$ . A contradiction.

∎

Ordinal potential.

To show an ordinal potential, we use the following definitions:

For a configuration $s \in S$ in a game $G_{Π, C, F}$ , we define $l i s t (s)$ to be the sequence of pairs in ${⟨ R P U_{c} (s), c ⟩ ∣ c \in C}$ ordered lexicographically from smallest to largest. Denote by $v_{i} (s)$ the coin (second element of the pair) in the $i^{t h}$ entry in $l i s t (s)$ . Consider the ordered set $⟨ L, ≺_{L} ⟩$ , where $L ≜ {l i s t (s) ∣ s \in S}$ is the set of all possible lists in $G_{Π, C, F}$ , and $≺_{L}$ is the lexicographical order. The rank of a list $l i s t (s) \in L$ , $r a n k (l i s t (s))$ , is the rank of $l i s t (s)$ in $≺_{L}$ from smallest to largest.

Note that since $Π$ and $C$ are finite, we know that $S$ and $L$ are finite. The following two observations establish a connection between better response steps and the $R P U s$ of the associated coins.

Observation 1.

Consider a game $G_{Π, C, F}$ , $s \in S$ , $v_{i} (s) \in C$ , and $p \in Π$ s.t. $s . p = v_{i} (s)$ . Then in every better response step of $p$ that changes $s . p$ to a coin $v_{j} (s)$ , it holds that $j > i$ .

Observation 2.

Consider a game $G_{Π, C, F}$ . If some better response step from configuration $s$ to configuration $s^{'}$ of a miner $p$ changes $s . p = c$ to $s^{'} . p = c^{'}$ , then $R P U_{c} (s) < m i n (R P U_{c} (s^{'}), R P U_{c^{'}} (s^{'}))$ .

We are now ready to prove that any game $G_{Π, C, F}$ has an ordinal potential function.

Theorem 1.

For any finite sets $Π$ and $C$ of miners and coins and reward function $F$ , $H (s) ≜ r a n k (l i s t (s))$ is an ordinal potential in the game $G_{Π, C, F}$ .

Proof.

Consider two configurations $s, s^{'} \in S$ s.t. some better response step of a miner $p \in Π$ leads from configuration $s$ to configuration $s^{'}$ , and let $v_{i} (s) = s . p$ and $v_{j} (s) = s^{'} . p$ . We need to show that $H (s) < H (s^{'})$ . Since only the RPUs of $v_{i} (s)$ and $v_{j} (s)$ are affected we get that

\forall k \neq i, j R P U_{v_{k} (s)} (s) = R P U_{v_{k} (s)} (s^{'}) .

(1)

By Observation 1, we get that $j > i$ , and thus $\forall k, 1 \leq k < i$ , $R P U_{v_{k} (s)} (s) = R P U_{v_{k} (s)} (s^{'}) .$ By Observation 2, we get that $m i n (R P U_{v_{i} (s)} (s^{'}), R P U_{v_{j} (s)} (s^{'})) > R P U_{v_{i} (s)} (s)$ , and thus, together with the definition of $v_{i}$ and Equation 1, we get that $\forall k, i \leq k \leq | C |$ , $R P U_{v_{k} (s)} (s^{'}) \geq R P U_{v_{i} (s)} (s) .$ Therefore, none of them “move down” to a position before $i$ in $l i s t (s^{'})$ and so

\forall k, 1 \leq k < i, ⟨ R P U_{v_{k} (s)} (s), v_{k} (s) ⟩ = ⟨ R P U_{v_{k} (s^{'})} (s^{'}), v_{k} (s^{'}) ⟩ .

(2)

That is, the first $i - 1$ elements of $l i s t (s)$ are equal to the first $i - 1$ elements of $l i s t (s^{'})$ . Hence, it suffices to show that the $i^{t h}$ element of $l i s t (s^{'})$ is lexicographically larger than the $i^{t h}$ element of $l i s t (s)$ . Let $v_{l} (s) = v_{i} (s^{'})$ . From Equation 2, we know that $l \geq i$ , so there are two possible cases:

First, $l \in {i, j}$ . The theorem follows from Observation 2.

Second, $l > i, l \neq j$ . In this case,

$⟨ R P U_{v_{i} (s)} (s), v_{i} (s) ⟩$	$< ⟨ R P U_{v_{l} (s)} (s), v_{l} (s) ⟩$	$(lexicographical order of l i s t (s))$
	$= ⟨ R P U_{v_{l} (s)} (s^{'}), v_{l} (s) ⟩$	$(Equation~{} ???)$
	$= ⟨ R P U_{v_{i} (s^{'})} (s^{'}), v_{i} (s^{'}) ⟩$	$(Definition of v_{l} (s)),$

as needed.

∎

4 There is often a better equilibrium

Before moving to our second major result in which we describe a manipulation through dynamic reward design that transitions the system between equilibria, in this section we show that under broad circumstances, in every stable configuration there is at least one miner who has higher payoff in another stable configuration. This means that such a miner will gain from moving the system there. Specifically, we prove this for games that satisfy the following assumptions (note that we use these assumptions only in this section):

Assumption 1 (Never alone).

For a configuration $s \in S$ in a game $G_{Π, C, F}$ , if there is a coin $c \in C$ s.t. $| P_{c} (s) | \leq 1$ , then there is a miner $p \in Π$ s.t. changing $s . p$ to $c$ is a better response step for $p$ .

Although this assumption cannot hold when $| Π | < 2 | C |$ , it often holds in practice since the number of miners must be much larger than the number of coins for the cryptocurrency to be secure (truly decentralized).

Assumption 2 (Generic game).

For any two coins $c \neq c^{'} \in C$ and two sets of players $P, P^{'} \subseteq Π$ in a game $G_{Π, C, F}$ , $\frac{F (c)}{Σ_{p \in P} m_{p}} \neq \frac{F (c^{'})}{Σ_{p \in P^{'}} m_{p}}$ .

This assumption is common in game theory

DBLP:journals/mss/HolzmanL03 , and it makes sense in our game since mining power in practice is measured in billions of operations per hour and coin rewards are coupled with coin fiat exchange rates, so exact equality is unlikely.

The following observation follows from Assumption 1 and the fact that coins that are chosen by at least one miner always divide their entire reward. It stipulates that in every stable configuration, the sum of the payoffs the miners get is equal to the sum of the coins’ rewards.

Observation 3 (All stable configurations are globally optimal).

For every stable configuration $s \in S$ in a game $G_{Π, C, F}$ under Assumption 1, it holds that $\sum_{p \in Π} u_{p} (s) = \sum_{c \in C} F (c)$ .

From Observation 3 and Assumption 2 it is easy to show the following claim:

Claim 4.

Consider a game $G_{Π, C, F}$ under Assumptions 1 and 2. If the game has more than one stable configuration, then for every stable configuration $s$ there exist a miner $p$ and a stable configuration $s^{'}$ s.t. $u_{p} (s^{'}) > u_{p} (s)$ .

It remains to show that $G_{Π, C, F}$ has more than one stable configuration. Consider $Π = {p_{1}, \dots, p_{n}}$ s.t. $m_{p_{1}} \geq m_{p_{2}} \geq \dots \geq m_{p_{n}}$ , and $\forall i, 1 \leq i \leq n$ , let $Π_{i} = {p_{1}, \dots, p_{i}}$ . We first show that the game $G_{Π_{2}, C, F}$ has two different configurations in which miners $p_{1}, p_{2}$ do not share a coin and at most one of them is unstable. Then, we inductively construct two configurations in $G_{Π_{i}, C, F}$ , $\forall i, 3 \leq i \leq n$ , based on the two configurations in $G_{Π_{i - 1}, C, F}$ , in which all miners in $Π_{i - 1}$ keep their locations and all miners except maybe the one that was unstable in $G_{Π_{1}, C, F}$ are stable. The construction step is captured by Claim 5, where $p_{n e w} = p_{i}$ in the $i^{t h}$ step.

Claim 5.

Let $F$ be a reward function. Consider a system $Q = ⟨ Π, C ⟩$ , and a configuration $s \in S_{Q}$ . Now consider another system $Q^{'} = ⟨ Π^{'}, C ⟩$ s.t. $Π^{'} = Π \cup {p_{n e w}}$ , $p_{n e w} \notin Π$ , and $m_{p_{n e w}} \leq m i n {m_{p} | p \in Π}$ . Let $c = argmax c^{'} \in C F (c^{'}) \frac{m_{p_{n e w}}}{M_{c^{'}} (s) + m_{p_{n e w}}}$ and consider a configuration $s^{'} \in S_{Q^{'}}$ s.t. for all $p \in Π$ $s^{'} . p = s . p$ and $s^{'} . p_{n e w} = c$ . Then $p_{n e w}$ is stable in $s^{'}$ in game $G_{Π^{'}, C, F}$ , and every player $p \in Π$ that is stable in $s$ in $G_{Π, C, F}$ is also stable in $s^{'}$ in $G_{Π^{'}, C, F}$ .

Finally, we show that the two configurations we construct in $G_{Π, C, F}$ are stable: Let $p_{n s}$ be the (possibly) unstable miner. By Assumption 1 (note that the assumption refers only to game $G_{Π, C, F}$ ), $p_{n s}$ cannot be alone in a coin (otherwise there must be another unstable miner), and thus it shares the coin with a smaller stable miner, which we show implies that $p_{n s}$ is stable.

Our results are captured by the following proposition, which follows from Claim 4 and and the inductive construction using Claim 5.

Proposition 2.

Consider a game $G_{Π, C, F}$ under Assumptions 1 and 2. Then for every stable configuration $s$ in $G_{Π, C, F}$ there exist a miner $p$ and a stable configuration $s^{'} \neq s$ in which $u_{p} (s^{'}) > u_{p} (s)$ .

5 Reward design: moving between equilibria

In this section we consider a system $Q = ⟨ Π, C ⟩$ , where $Π = {p_{1}, \dots p_{n}}$ s.t. $m_{p_{1}} > m_{p_{2}} > \dots > m_{p_{n}}$ . For every reward function $F$ and every two stable configurations $s_{0}, s_{f} \in S_{Q}$ in game $G_{Π, C, F}$ we describe a mechanism to move the system from $s_{0}$ to the desired configuration $s_{f}$ by temporarily increasing coin rewards. Note that once we lead the system to $s_{f}$ , we can return to the original rewards (i.e., stop manipulating coin weights) because $s_{f}$ is stable in $G_{Π, C, F}$ . Therefore, a manipulator who gains from moving to a desired stable configuration can do it with a bounded cost.

We first define a reward design function that maps system configurations to reward functions.

Definition 1 (reward design function).

Consider a system $Q$ . A reward design function $F$ for system $Q$ is a function mapping every configuration $s \in S_{Q}$ to a reward function, i.e., $F (s) : C \to R_{+}$ .

Dynamic reward design.

Consider a system $⟨ Π, C ⟩$ and a reward function $F$ . A dynamic reward design mechanism for game $G_{Π, C, F}$ is an algorithm that for any two stable configurations $s_{0}, s_{f}$ in $G_{Π, C, F}$ moves the system from $s_{0}$ to $s_{f}$ by following the protocol in Algorithm 1.

s \leftarrow

s_{0}

2:repeat

3: choose a reward design function

H

s.t. for all

c \in C

H (s) (c) \geq F (c)

4: allow better-response learning in

G_{Π, C, H (s)}

, starting from

s

, to converge to some stable

5: configuration

s^{'}

▹

convergence is due to Theorem 1

s \leftarrow

s^{'}

7:until

s = s_{f}

Algorithm 1 protocol to move a system

⟨ Π, C ⟩

with reward function

F

from

s_{0}

s_{f}

5.1 Reward design algorithm

To describe a dynamic reward design algorithm we need to specify the reward design function for every loop iteration in Algorithm 1. Intuitively, we observe that miners with less mining power are easily moved between coins, meaning that we can increase a coin reward so that a small miner with little mining power will benefit from moving there, but bigger miners with more mining power prefer to stay in their current locations. Therefore, the idea is to evolve the current configuration to $s_{f} \in S$ in $n = | Π |$ stages, where in stage $i$ , we move the $n - i + 1$ miners with the smallest mining powers to the location (coin) of miner $p_{i}$ in the final configuration $s_{f}$ (i.e., $s_{f} . p_{i}$ ) while keeping the remaining miners in their (final) places. To this end, we define $n$ intermediate configurations. For $i$ , $1 \leq i \leq n$ , we define $s_{i}$ as:

s_{i} . p_{k} = {\begin{matrix} s_{f} . p_{k} & \forall 1 \leq k \leq i s_{f} . p_{i} & \forall i < k \leq n \end{matrix}

(3)

That is, in $s_{i}$ , miners $p_{1}, \dots, p_{i}$ are in their final locations and miners $p_{i}, \dots, p_{n}$ are in the final location of miner $p_{i}$ . Note that $s_{n} = s_{f}$ . Figure 1(a) illustrates the stage transitions in the algorithm.

Notice that since we allow arbitrary better response learning (in every iteration), choosing a reward design function is a subtle task; miners can move according to any better response step, and we cannot control the order in which miners move. One may attempt to design a reward function so that in the resulting game there is exactly one unstable miner with exactly one better response step in the current configuration. However, even given such a function, after that miner takes its step, other miners might become unstable, which can in turn lead to a learning process that depends on the order in which miners move and on the choices they make (in case they have more than one better response step). Hence, the main challenge is to be able to restrict the set of the possible stable configurations reached by learning phase in each iteration.

In every loop iteration of stage $i > 1$ we pick a miner $p_{k}$ that we want to move from $s_{f} . p_{i - 1}$ to $s_{f} . p_{i}$ (as explained shortly) and choose the reward function carefully so that (1) $p_{k}$ ’s only better response step is $s_{f} . p_{i}$ , (2) all other miners are stable, and (3) in every stable configuration reached by better response learning after $p_{k}$ ’s step, $p_{k}$ is in $s_{f} . p_{i}$ , all miners $p_{k + 1}, \dots, p_{n}$ are in either $s_{f} . p_{i - 1}$ or $s_{f} . p_{i}$ , and all the other (bigger) miners remain in their (final) locations.

Moreover, our proof shows by induction that our reward design function of stage $i$ (defined below) guarantees that the set of possible configurationas reached by learning in stage $i > 1$ is

T_{i} ≜ {s \in S ∣ (\forall k, 1 \leq k \leq i - 1 : s . p_{k} = s_{f} . p_{k}) \land (\forall k, i \leq k \leq n : s . p_{k} \in {s_{f} . p_{i}, s_{f} . p_{i - 1})}} .

Notice that the stage starts at $s_{i - 1} \in T_{i}$ . We now explain how we choose the reward design function for stage $i$ . First, for every configuration $s \in T_{i} ∖ {s_{i}}$ , the index of the miner we want to move from $s_{f} . p_{i - 1}$ to $s_{f} . p_{i}$ (called mover) is

m_{i} (s) = m i n {j | \forall l, j < l \leq n : s . p_{l} = s_{f} . p_{i}} .

Note that for every $s \in T_{i}$ , $i \leq m_{i} (s) \leq n$ . Moreover, $p_{m_{i} (s)} \notin P_{s_{f} . p_{i}} (s)$ and $m_{i} (s_{i - 1}) = n$ . Let $a_{i} (s) = m_{i} (s) - 1$ . Intuitively, we use $p_{a_{i} (s)}$ as an anchor in configuration $s$ ; we choose a reward function that increases the reward of coin $s_{f} . p_{i}$ as high as possible without making the anchor unstable. As a result, all the miners in $P_{s_{f} . p_{i - 1}} (s)$ (who are bigger than or equal to the anchor) remain stable, and miner $p_{m_{i} (s)}$ has a unique better response step to move to $s_{f} . p_{i}$ . Figure 1(b) illustrates $m_{i} (s)$ and $a_{i} (s)$ for some configuration $s \in T_{i}$ .

In order to make sure that miners not in $P_{s_{f} . p_{i - 1}} (s) \cup P_{s_{f} . p_{i}} (s)$ also remain stable, and in order to guarantee that that every better response learning after $p_{m_{i} (s)}$ ’s step converges to a configuration in $T_{i}$ , we choose a reward function that evens out the RPUs of all coins other than $s_{f} . p_{i}$ . For $s \in S$ , let $R (s) = m a x {R P U_{c} (s) ∣ c \in C}$ $\forall s \in S$ . The reward design function $H_{i}$ for stage $i > 1$ is:

\forall i > 1 \forall s \in T_{i} \forall c \in C, H_{i} (s) (c) = {\begin{matrix} R (s) \cdot (M_{c} (s) + m_{p_{a_{i} (s)}}) & for c = s_{f} . p_{i} R (s) \cdot M_{c} (s) & otherwise \end{matrix}

(4)

Note that the RPUs of all coins except $s_{f} . p_{i}$ in the game $G_{Π, C, H_{i} (s)}$ are equal to $R (s)$ . In addition, note that if a miner bigger than or equal to $p_{a_{i} (s)}$ moves to $s_{f} . p_{i}$ , then $s_{f} . p_{i}$ ’s RPU becomes no bigger than $R (s)$ . However, since $m_{p_{m_{i} (s)}} < m_{p_{a_{i} (s)}}$ , $p_{m_{i} (s)}$ has a unique better response step to move to $s_{f} . p_{i}$ . Therefore, we get that our reward design function allows us to control the first step of the learning process. In the next section we give more intuition on how it also restricts the stable configuration at the end of any learning process at stage $i$ to the set $T_{i}$ .

As for the fist stage, note that we need to move all miners to coin $s_{f} . p_{1}$ , so intuitively we only need to increase its reward high enough. We therefore choose:

\forall s \in S \forall c \in C, H_{1} (s) (c) = {\begin{matrix} m a x {F (c^{'}) ∣ c^{'} \in C} \cdot Σ_{p \in Π} m_{p} & for c = s_{f} . p_{1} F (c) & otherwise \end{matrix}

(5)

In Algorithm 2 we present our reward design algorithm, and in the next section we outline the proof that every stage eventually completes.

s \leftarrow

s_{0}

2:for i=1 …n do

3: repeat

▹

H_{i}

is defined in Equations 4 and 5

4: allow better-response learning in

G_{Π, C, H_{i} (s)}

, starting from

s

, to converge to some stable

5: configuration

s^{'}

s \leftarrow

s^{'}

7: until

s = s_{i}

▹

s_{i}

is defined in Equations 3

Algorithm 2 Dynamic reward design algorithm.

5.2 Proof outline

The proof for stage 1 is straightforward so we skip it. Consider stage $i > 1$ . We prove in the appendix the following technical lemma about stable configurations in the stage:

Lemma 1.

Consider a configuration $s \in T_{i} ∖ {s_{i}}$ . Then every better response learning in the game $G_{Π, C, H_{i} (s)}$ that starts at $s$ converges to a configuration $s^{'} \in T_{i}$ such that:

$\forall k, 1 \leq k < m_{i} (s)$ , $s^{'} . p_{k} = s . p_{k}$ .
$s^{'} . p_{m_{i} (s)} = s_{f} . p_{i}$ .

As part of the proof, we show that within stage $i$ , all the reached configurations (both stable and unstable) are in $T_{i}$ . Let $c = s_{f} . p_{i - 1}$ and $c^{'} = s_{f} . p_{i}$ . After $p_{m_{i} (s)}$ moves to $c^{'}$ according to its only better response step, in the resulting configuration $s^{'}$ , the RPUs of all coins not in ${c, c^{'}}$ remain $R (s)$ . Moreover, $R P U_{c} (s^{'}) = \frac{H_{i} (c)}{M_{c} (s^{'})} = \frac{R (s) \cdot M_{c} (s)}{M_{c} (s) - m_{p_{m_{i} (s)}}}$ , and $R P U_{c^{'}} (s^{'}) = \frac{H_{i} (c^{'})}{M_{c^{'}} (s^{'})} = \frac{R (s) \cdot (M_{c} (s) + m_{p_{a_{i} (s)}})}{M_{c} (s) + m_{p_{m_{i} (s)}}}$ . Therefore, although $R P U_{c} (s^{'}) > R P U_{c} (s)$ , it is still not high enough to drive miners not in $P_{c^{'}} (s^{'})$ (by definition, bigger than $p_{m_{i} (s)}$ ) to move to it. So the only miners that possibly have better response steps at $s^{'}$ are miners in $P_{c^{'}} (s^{'})$ who wish to move to $c$ . Moreover, the total mining power of the miners who actually move to $c$ is smaller than $p_{m_{i} (s)}$ , otherwise, $c$ ’s RPU will go below $R (s)$ . In the proof we use the above intuition to formulate an invariant that captures the lemma statement and prove it by induction on better response steps. The lemma then follows from Theorem 1 (every better response learning converges to a stable configuration).

We next use Lemma 1 to prove that every stage $i > 1$ completes in a finite number of loop iterations. To this end, we associate with every configuration $s T_{i}$

a binary vector

v (s)

indicating, for each

j \geq i

, whether

p_{j}

is in

P_{s_{f} . p_{i}} (s)

, where it needs to be at the end of the stage. Consider the ordered set

⟨ V, ≺_{v} ⟩

, where

V ≜ {0, 1}^{n - i + 1}

is the set of all binary vectors of length

n - i + 1

, and

≺_{v}

is the lexicographical order. For a configuration

s \in T_{i}

, we define

v e c (s)

to be a vector in

V

such that:

\forall j, 1 \leq j \leq n - i + 1, v e c (s) [j] = {\begin{matrix} 1 & if p_{j + i - 1} \in P_{s_{f} . p_{i}} (s) 0 & otherwise \end{matrix}

and the function $Φ_{i} : T_{i} \to {1, . ., | V |}$ to be the rank of $v e c (s)$ in $V$ .

Theorem 2.

Every stage $i > 1$ of Algorithm 2 completes in a finite number of loop iterations.

Proof.

By definitions of stage $i$ and set $T_{i}$ , the first configuration of stage $i$ is $s_{i - 1} \in T_{i}$ . By definition of $m_{i} (s)$ , for all $s \in T_{i} ∖ {s_{i}}$ , $m_{i} (s) \notin P_{s_{f} . p_{i}} (s)$ . By inductively applying Lemma 1, we get that every loop iteration in stage $i$ ends in a configuration in $T_{i}$ . Therefore, consider a loop iteration of stage $i$ that starts in configuration $s \neq s_{i}$ and ends in configuration $s^{'}$ , we get by Lemma 1 that $m_{i} (s) \in P_{s_{f} . p_{i}} (s^{'})$ and $\forall k, i \leq k < m_{i} (s)$ , $s . p_{k} = s^{'} . p_{k}$ . Therefore $Φ (s^{'}) > Φ (s)$ . Now since the set $T_{i}$ is finite, we get the after a finite number of iterations we reach configuration $s_{i}$ .

∎

6 Discussion

Our work studies and challenges the crypocurrency market from a novel angle – the strategic selections by adaptive miners among multiple coins. There are several central followups one may consider. First, our reward design is effective for arbitrary better-response learning, but one may wonder about its speed of convergence under specific markets. In addition, we consider convergence to equilibrium, and one may consider also convergence to a bad (possibly unstable) configuration in which, for example, a particular miner will have a dominant position in a coin, killing (at least for a while) the basic guarantee of non-manipulation (security) for that coin and allowing him to get a bigger portion of the reward. One also may wonder about the asymmetric case where some coins can be mined only by a subset of the miners.

References

[1] Bitcoin cash. https://www.bitcoincash.org/. Accessed: 2018-04-23.
[2] Bitcoin price manipulation: Economists warn just one person may have caused value surge. https://www.express.co.uk/finance/city/905222/bitcoin-price-news-latest-manipulation-cryptocurrency-surge-plummet-stock-exchange. Accessed: 2018-04-24.
[3] Cardano. https://www.cardano.org/en/home/. Accessed: 2018-04-23.
[4] Crypto whales and how they manipulate the price. https://steemit.com/cryptocurrency/@endpoint/crypto-whales-and-how-they-manipulate-the-price. Accessed: 2018-04-24.
[5] Cryptocurrency info charts. https://bitinfocharts.com/comparison/hashrate-btc-bch.html#6m. Accessed: 2018-04-23.
[6] Cryptocurrency market state visualization. https://coin360.io/. Accessed: 2018-04-23.
[7] Cryptocurrency price manipulation is unavoidable. https://www.cnbc.com/2018/02/13/cryptocurrency-price-manipulation-is-unavoidable-nem-president-says.html. Accessed: 2018-04-24.
[8] Litecoin. https://litecoin.com/. Accessed: 2018-04-23.
[9] Neo. https://neo.org/. Accessed: 2018-04-23.
[10] What to mine. https://whattomine.com/. Accessed: 2018-04-24.
[11] Vitalik Buterin et al. A next-generation smart contract and decentralized application platform.
[12] Miles Carlsten, Harry Kalodner, S Matthew Weinberg, and Arvind Narayanan. On the instability of bitcoin without the block reward. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, pages 154–167. ACM, 2016.
[13] Nicolò Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge University Press, 2006.
[14] Ittay Eyal. The miner’s dilemma. In Security and Privacy (SP), 2015 IEEE Symposium on, pages 89–103. IEEE, 2015.
[15] Ittay Eyal and Emin Gun Sirer. Majority is not enough: Bitcoin mining is vulnerable. In International Conference on Financial Cryptography and Data Security, pages 436–454. Springer, 2014.
[16] Neil Gandal, JT Hamrick, Tyler Moore, and Tali Oberman. Price manipulation in the bitcoin ecosystem. Journal of Monetary Economics, 2018.
[17] Dylan Hadfield-Menell, Smitha Milli, Pieter Abbeel, Stuart J Russell, and Anca Dragan. Inverse reward design. In Advances in Neural Information Processing Systems, pages 6768–6777, 2017.
[18] Amélie Heliou, Johanne Cohen, and Panayotis Mertikopoulos. Learning with bandit feedback in potential games. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 6369–6378. Curran Associates, Inc., 2017.
[19] Ron Holzman and Nissan Law-yone. Network structure and strong equilibrium in route selection games. Mathematical Social Sciences, 46(2):193–205, 2003.
[20] Benjamin Johnson, Aron Laszka, Jens Grossklags, Marie Vasek, and Tyler Moore. Game-theoretic analysis of ddos attacks against bitcoin mining pools. In International Conference on Financial Cryptography and Data Security, pages 72–86. Springer, 2014.
[21] Aggelos Kiayias, Elias Koutsoupias, Maria Kyropoulou, and Yiannis Tselekounis. Blockchain mining games. In Proceedings of the 2016 ACM Conference on Economics and Computation, pages 365–382. ACM, 2016.
[22] Kevin Liao and Jonathan Katz. Incentivizing blockchain forks via whale transactions. In International Conference on Financial Cryptography and Data Security, pages 264–279. Springer, 2017.
[23] Igal Milchtaich. Congestion games with player-specific payoff functions. Games and Economic Behavior, 13, 1996.
[24] D. Monderer and L.S. Shapley. Potential games. Games and Economic Behavior, 14:124–143, 1996.
[25] Satoshi Nakamoto. Bitcoin: A peer-to-peer electronic cash system, 2008.
[26] Kartik Nayak, Srijan Kumar, Andrew Miller, and Elaine Shi. Stubborn mining: Generalizing selfish mining and combining with an eclipse attack. In 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pages 305–320. IEEE, 2016.
[27] Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani. Algorithmic Game Theory. Cambridge University Press, New York, NY, USA, 2007.
[28] Gerasimos Palaiopanos, Ioannis Panageas, and Georgios Piliouras. Multiplicative weights update with constant step-size in congestion games: Convergence, limit cycles and chaos. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 5874–5884, 2017.
[29] Ayelet Sapirshtein, Yonatan Sompolinsky, and Aviv Zohar. Optimal selfish mining strategies in bitcoin. In International Conference on Financial Cryptography and Data Security, pages 515–532. Springer, 2016.
[30] Okke Schrijvers, Joseph Bonneau, Dan Boneh, and Tim Roughgarden. Incentive compatibility of bitcoin mining pool reward functions. In International Conference on Financial Cryptography and Data Security, pages 477–498. Springer, 2016.
[31] Jonathan Sorg, Richard L Lewis, and Satinder P. Singh. Reward design via online gradient ascent. In J. D. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 2190–2198. Curran Associates, Inc., 2010.
[32] Jonathan Sorg, Satinder P. Singh, and Richard L. Lewis. Internal rewards mitigate agent boundedness. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel, pages 1007–1014, 2010.
[33] Richard S. Sutton and Andrew G. Barto. Reinforcement learning - an introduction. Adaptive computation and machine learning. MIT Press, 1998.
[34] Vasilis Syrgkanis, Alekh Agarwal, Haipeng Luo, and Robert E Schapire. Fast convergence of regularized learning in games. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 2989–2997. Curran Associates, Inc., 2015.

Appendix A Existence of an equilibrium

We show here how to find a stable configuration (pure equilibrium) in the game $G_{Π, C, F}$ for any $Π, C$ , and $F$ . We do this by induction, selecting coins for miners in descending order of mining power.

Claim 6.

Consider a reward function $F$ , a system $Q = ⟨ Π, C ⟩$ , and another system $Q^{'} = ⟨ Π^{'}, C ⟩$ s.t. $Π^{'} = Π \cup {p_{n e w}}$ , $p_{n e w} \notin Π$ , and $m_{p_{n e w}} \leq m i n {m_{p} | p \in Π}$ . Then, if the game $G_{Π, C, F}$ has a stable configuration, than the game $G_{Π^{'}, C, F}$ has a stable configuration as well.

Proof.

Let $s$ be a stable configuration in $G_{Π, C, F}$ . We use it to build a configuration $s^{'}$ in $G_{Π^{'}, C, F}$ in the following way: for all $p \neq p_{n e w}$ set $s^{'} . p = s . p$ , and set $s^{'} . p_{n e w}$ to $c = argmax c^{'} \in C F (c^{'}) \frac{m_{p_{n e w}}}{M_{c^{'}} (s) + m_{p_{n e w}}}$ . We now show that configuration $s^{'}$ is stable in $G_{Π^{'}, C, F}$ .

First, consider $p_{n e w}$ . Since we pick $c$ to be $argmax c^{'} \in C F (c^{'}) \frac{m_{p_{n e w}}}{M_{c^{'}} (s) + m_{p_{n e w}}}$ , we get that $\forall c^{'} \in C$ , $F (c) \frac{m_{p_{n e w}}}{M_{c} (s^{'})} \geq F (c^{'}) \frac{m_{p_{n e w}}}{M_{c^{'}} (s^{'}) + m_{p_{n e w}}}$ , so $p_{n e w}$ is stable in $s^{'}$ . Next, consider a miner $p$ s.t. $s . p = c^{'} \neq c$ . Since $s$ is stable, we know that $\forall c^{''} \in C$ , $F (c^{'}) \frac{m_{p}}{M_{c^{'}} (s)} \geq F (c^{''}) \frac{m_{p}}{M_{c^{''}} (s) + m_{p}}$ . Now since $M_{c} (s) < M_{c} (s^{'})$ and for all $c^{''} \neq c$ $M_{c^{''}} (s) = M_{c^{''}} (s^{'})$ , we get that $\forall c^{''} \in C, F (c^{'}) \frac{m_{p}}{M_{c^{'}} (s^{'})} \geq F (c^{''}) \frac{m_{p}}{M_{c^{''}} (s^{'}) + m_{p}}$ . Meaning that $p$ is stable in $s^{'}$ .

Finally, consider $p \neq p_{n e w}$ s.t. $s^{'} . p = c$ . We need to show that for all $c^{'} \neq c$ , $F (c^{'}) \frac{m_{p}}{M_{c^{'}} (s^{'}) + m_{p}} \leq F (c) \frac{m_{p}}{M_{c} (s^{'})}$ . Again, since we pick $c$ to be $argmax c^{'} \in C F (c^{'}) \frac{m_{p_{n e w}}}{M_{c_{j}} (s) + m_{p_{n e w}}}$ , we know that $\forall c^{'} \neq c$ , $F (c) \frac{m_{p_{n e w}}}{M_{c} (s) + m_{p_{n e w}}} \geq F (c^{'}) \frac{m_{p_{n e w}}}{M_{c^{'}} (s) + m_{p_{n e w}}}$ , and thus $\forall c^{'} \neq c$ , $F (c) \frac{1}{M_{c} (s^{'})} \geq F (c^{'}) \frac{1}{M_{c^{'}} (s^{'}) + m_{p_{n e w}}}$ . Now by the claim assumption, $m_{p_{n e w}} \leq m_{p}$ . Therefore, $\forall c^{'} \neq c$ , $F (c) \frac{1}{M_{c} (s^{'})} \geq F (c^{'}) \frac{1}{M_{c^{'}} (s^{'}) + m_{p_{n e w}}} \geq F (c^{'}) \frac{1}{M_{c^{'}} (s^{'}) + m_{p}}$ . Thus, $\forall c^{'} \neq c$ , $F (c) \frac{m_{p}}{M_{c} (s^{'})} \geq F (c^{'}) \frac{m_{p}}{M_{c^{'}} (s^{'}) + m_{p}}$ . Meaning that $p$ is stable in $s^{'}$ .

∎

Proposition 3.

For any set of of miners $Π = {p_{1}, \dots, p_{n}}$ , set of coins $C$ , and a reward function $F$ , the game $G_{Π, C, F}$ has a stable configuration.

Proof.

Order miners by mining power so that $m_{p_{1}} \geq m_{p_{2}} \geq \dots \geq m_{p_{n}}$ . First we show that the game $G_{{p_{1}}, C, F}$ has a stable configuration. Let $c = argmax c^{'} \in C F (c^{'})$ , and define a configuration $s$ in $G_{{p_{1}}, C, F}$ as follows: $s . p_{1} = c$ . Since $\forall c^{'} \in C, F (c) \frac{m_{p_{1}}}{M_{c} (s)} = F (c) \geq F (c^{'}) = F (c^{'}) \frac{m_{p_{1}}}{M_{c^{'}} (s) + m_{p_{1}}}$ , we get that the configuration $s$ is stable in $G_{{p_{1}}, C, F}$ . The lemma follows by inductively applying Claim 6.

∎

Appendix B Ordinal potential for the symmetric case

We consider here the symmetric case where all coin rewards are equal, and show that any game in this case has a simple potential function.

Game of Coins

Related Research

Game Networks

Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning

Inverse Game Theory for Stackelberg Games: the Blessing of Bounded Rationality

Bounded strategic reasoning explains crisis emergence in multi-agent market games

On Liquidity Mining for Uniswap v3

Human-Agent Decision-making: Combining Theory and Practice

Convergence of Learning Dynamics in Information Retrieval Games

Code Repositories

gameofcoins

1 Introduction

1.1 Related work

2 Model

3 Better response learning convergance

No exact potential.

Proposition 1.

Proof.

Ordinal potential.

Observation 1.

Observation 2.

Theorem 1.

Proof.

4 There is often a better equilibrium

Assumption 1 (Never alone).

Assumption 2 (Generic game).

Observation 3 (All stable configurations are globally optimal).

Claim 4.

Claim 5.

Proposition 2.

5 Reward design: moving between equilibria

Definition 1 (reward design function).

Dynamic reward design.

5.1 Reward design algorithm

5.2 Proof outline

Lemma 1.

Theorem 2.

Proof.

6 Discussion

References

Appendix A Existence of an equilibrium

Claim 6.

Proof.

Proposition 3.

Proof.

Appendix B Ordinal potential for the symmetric case

Proposition 4.