Smoothed Complexity of 2-player Nash Equilibria

07/21/2020 ∙ by Shant Boodaghians, et al. ∙ 0

We prove that computing a Nash equilibrium of a two-player (n × n) game with payoffs in [-1,1] is PPAD-hard (under randomized reductions) even in the smoothed analysis setting, smoothing with noise of constant magnitude. This gives a strong negative answer to conjectures of Spielman and Teng [ST06] and Cheng, Deng, and Teng [CDT09]. In contrast to prior work proving PPAD-hardness after smoothing by noise of magnitude 1/poly(n) [CDT09], our smoothed complexity result is not proved via hardness of approximation for Nash equilibria. This is by necessity, since Nash equilibria can be approximated to constant error in quasi-polynomial time [LMM03]. Our results therefore separate smoothed complexity and hardness of approximation for Nash equilibria in two-player games. The key ingredient in our reduction is the use of a random zero-sum game as a gadget to produce two-player games which remain hard even after smoothing. Our analysis crucially shows that all Nash equilibria of random zero-sum games are far from pure (with high probability), and that this remains true even after smoothing.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Nash equilibrium is the central solution concept in game theory. Computational complexity results establishing the intractability of Nash equilibrium 

[SV06, CDT09, DGP09] suggest that players that are even mildly computationally bounded may not be able to converge to a Nash equilibrium in the worst case. However, the fragility of these intractable game constructions, together with the fact that random games are tractable [BVV07], have led experts to conjecture that Nash equilibrium should have smoothed polynomial time algorithms (e.g. [ST06] Conjecture 15, and [CDT09] Conjecture 2). If these conjectures were true, they could explain why players in realistic games can converge to equilibrium. In this paper, we prove that even with aggressive smoothing perturbations of constant magnitude, finding a Nash equilibrium continues to be -complete (under randomized reductions).

Definition 1 (-Smoothed-Nash).

For a distribution on and problem size , fix worst-case matrices with entries in , and let be matrices whose entries are drawn i.i.d. from . -SMOOTHED-NASH is the problem of computing, with probability111Amplifying the success probability of smoothed algorithms is generally non-trivial (and sometimes impossible). We note that our hardness result continues to hold even for algorithms that are only required to succeed with probability . at least , a Nash equilibrium of the game .

Theorem 1 (Main Theorem).

There exists a universal constant

, such that for any probability distribution

on , -SMOOTHED-NASH is -hard under a randomized reduction.222Formally, we assume that there is a (single-dimensional) distribution such that and can be sampled by randomized polynomial-time algorithm. This holds for any natural smoothing distribution – e.g. truncated Gaussian or uniform. For arbitrary such can be sampled by a randomized algorithm which receives as input a -size approximation of the CDF of . For where such advice is necessary, our arguments show that -SMOOTHED-NASH is -hard under randomized reductions with polynomial-length advice.

1.1 Complexity context: smoothed analysis vs hardness of approximation

In their 2006 survey on smoothed analysis, Spielman and Teng posed the challenge ([ST06], Open Question 11) of exploring the connections between smoothed complexity and hardness of approximation. Concretely, they considered the example of two-player Nash equilibrium subject to -bounded perturbations: Given a hard game , perturbing each entry independently gives rise to a new instance ; any Nash equilibrium of is an -approximate-Nash equilibrium of the original game . Hence, solving Nash equilibrium in the smoothed model is at least as hard as approximating Nash [ST06, Proposition 9.12].

More generally, any hard instance of any computational problem333Naturally, the correspondence between approximation algorithms and smoothed analysis requires matching the respective notions of approximation and smoothing perturbations. (e.g.  in the case of Nash) can be in one of three states (as illustrated in Figure 1):

Smoothed-algorithmica44footnotemark: 4:

Most instances in ’s neighborhood can be solved efficiently.

Smoothed-complexity:

A small fraction of ’s neighborhood can be solved efficiently.

Hardness-of-approximation:

Finding a solution for any instance in ’s neighborhood is intractable.666Intuitively we would like to say that every instance in ’s neighborhood is intractable. Formally, however, this may be inaccurate. In fact in the case of Nash equilibrium it is provably false! Given game , consider game , where is a matrix whose entries are identically equal to some small which encodes a Nash equilibrium for (and hence also for ).

Of course, as in Spielman and Teng’s proposition, hardness-of-approximation immediately rules out efficient smoothed algorithms. But most interesting open problems in smoothed analysis admit approximation algorithms; this limits the applicability of using hardness-of-approximation to prove new smoothed-complexity results.

In contrast to the thriving literature on hardness of approximation and smoothed algorithms, smoothed complexity results are rare. In this paper, we make a small step toward establishing a theory of smoothed complexity, in the context of Spielman and Teng’s original example: two-player Nash equilibrium subject to bounded perturbations777Speilman and Teng discuss perturbing each entry by a uniform- noise, but our result holds for any bounded i.i.d. perturbations. While settling an open problem in equilibrium computation, we believe that our result is just the tip of the iceberg of the theory of smoothed complexity.

1.2 Historical context

In 1928 Von Neumann [Neu28] proved that every (finite, perfect information) zero-sum game has an equilibrium; this result was extended to general games by Nash in 1951 [Nas51]. In 1947 Dantzig [Dan98]

designed the simplex algorithm for solving linear programs (and thus also zero-sum games); in 1964 Lemke and Howson 

[LH64] gave a simplex-like algorithm for general games. Both are known to take exponential time in the worst case [KM72, SV06], but are observed to perform much better in practice (e.g. [Sha87, ARSvS10]).

For linear programming, Khachiyan [Kha79] gave the first polynomial time algorithm in 1979, and Spielman and Teng proved in 2004 [ST04] that the simplex algorithm has smoothed polynomial complexity. It was natural to hope (and in fact quite widely believed, e.g. [DGP05] and [ST06, Conjecture 9.51] respectively) that the last two results would again be extended to general games. Surprisingly, this was ruled out by Chen, Deng, and Teng [CDT09]. Specifically, they showed that -approximate Nash equilibrium is hard, which by Spielman and Teng’s proposition rules out any smoothed efficient algorithms for noise magnitude (assuming is not contained in search-). Chen, Deng, and Teng nevertheless conjectured that for constant magnitude noise, two-player Nash equilibrium should have a polynomial time algorithm.

Progress on smoothed complexity of Nash with -noise (for small constant ) was made by [Rub16] who proved the following hardness of approximation result: assuming the “Exponential Time Hypothesis for 888The Exponential Time Hypothesis (ETH) for  is a strengthening of , which postulates that End-of-Line (the canonical -problem) requires time. ”, finding an -approximate Nash equilibrium requires quasipolynomial () time. By Spielman and Teng’s proposition, this hardness of approximation result also implies an analogous quasipolynomial hardness in the smoothed setting. For hardness of approximation, the result of [Rub16] is essentially optimal due to a matching quasipolynomial time approximation algorithm [LMM03]. This quasipolynomial time algorithm does not extend to the smoothed case, and a large gap in the complexity of constant-smoothed Nash (quasipolynomial vs exponential) remained open.

In this work, we resolve the complexity of two-player Nash equilibrium with constant-magnitude smoothing, proving that it is -complete (under randomized reductions). Compared to [Rub16], we rule out smoothed polynomial time algorithms under a much weaker assumption ( vs ETH for ).999As discussed above, this holds in the case that is approximately polynomial-time sampleable – otherwise we require the assumption . Alternatively, comparing both results under the same assumption, ETH for , we prove a much stronger lower bound on the running time ( vs )101010In fact, under the plausible hypothesis that the true complexity of End-of-Line (the canonical -problem) is for some constant , our result implies the qualitatively-same strong lower bound, and the result of  [Rub16] completely breaks.. Finally, another advantage of our result compared to [Rub16] is that our proof is much simpler, and in particular does not require any PCP-like machinery.

1.3 Intuition and roadmap

We will reduce -approximate Nash to -SMOOTHED-NASH. The starting point of our reduction is the following simple idea: for any mixed strategies which are spread over a large number of actions, the noise from the smoothing averages out. In contrast, if we start with an off-the-shelf -hard game

and amplify it by simple repetition (formally, tensor the payoff matrices

with the all ones matrix ), the signal from will remain strong even with respect to well-spread strategies. This means that given a well-spread (in a sense we make precise later) Nash equilibrium for a tensored, smoothed game , we can recover a -approximate equilibrium for .

There is one major problem with the reduction suggested above: an oracle for -SMOOTHED-NASH might not return a well-spread equilibrium . Our goal henceforth is to modify this construction to create a game where no Nash equilibrium has strategies concentrated on a small number of actions. Note that pure or even small-support equilibria don’t break only our proof approach: they can be found efficiently by brute-force enumeration, so such games cannot be hard.

Which games have no strategies concentrated on a small number of actions? At one extreme, if the entries of the payoff matrices are entirely i.i.d. (from any continuous distribution), a folklore result states that the game has a pure equilibrium with probability approaching . This creates a significant problem: we have to work with games where the entries are smoothed with independent noise – if such games turn out also to have pure or small-support strategies, then they cannot be hard.

In contrast to i.i.d. random games, we observe that random zero-sum games tend to have only well spread equilibria [Rob06, Jon04]. For example, they are exponentially unlikely to have a pure equilibrium; intuitively, if a pure strategy profile is exceptionally good for one player, it is likely exceptionally bad for the other. In the context of our proof approach, another advantage of random zero-sum games is that with respect to well-spread mixed strategies, they will also average out. That is, even if we add a random zero-sum game , we can still hope to recover a Nash equilibrium for from a well-spread equilibrium for . Our main technical task is to show that adding a random zero-sum game in this fashion produces a game with only well-spread Nash equilibria, even in the presence of the i.i.d. smoothing .

Our first step is to rule out all small support equilibria.

In Section 4 we formalize the above intuition, showing that every equilibrium of a random zero-sum game has large supports, even when we add constant-magnitude perturbations. For technical reasons, our proof in this section works for random zero-sum games whose entries are drawn uniformly from discrete .

Our second step is to obtain a robust version of no-small-support.

Namely, building on the fact every equilibrium has large support, in Section 5 we prove that it must be well-spread (formally, the mixed strategies have small norm). For technical reasons, our proof in this section works for random zero-sum games whose entries are drawn uniformly from continuous . Fortunately, we can make both of proofs work simultaneously by taking the sum of a and a zero-sum games.

Putting it all together.

To summarize, our final construction of hard instance is given by:

where are random matrices with i.i.d. entries uniformly sampled from and (respectively), and is a -hard bimatrix game, and is an (appropriate-dimension) all-ones matrix.

In Section 3, we show that when the Nash equilibrium strategies are well-spread, the random zero-sum games and random perturbations average out. Thanks to the amplification, the signal from remains sufficiently strong. Thus, we can map any Nash equilibrium of to a -approximate Nash equilibrium of . By [CDT09] this suffices to establish -hardness (under randomized reductions).

Remark (Inverse-polynomial signal-to-noise ratio).

Interestingly, the amplification of by repetition is so powerful that our proof would go through even if we were to multiply and by an inverse-polynomial small scalar.111111We only informally state the result to prioritize simplicity, but it will be evident by the remarks in Section 2.1. In this sense, we show that Nash remains intractable even subject to noise (zero-sum + i.i.d.) that is polynomially larger than the worst-case signal.

1.4 Additional related work

Subsequent to the seminal works of [DGP09, CDT09] which showed that Nash equilibrium is -complete, there has been an active line of work on algorithms with provable guarantees for exact or approximate equilibria in special cases including: sparse games [Bar18], low-rank games [KT10, AGMS11], positive-semidefinite games [ALSV13], anonymous games [DP15, CDS17], tree games [EGG06, BLP15, OI16]. Complexity limitations for most of these special cases known as well: sparse games [CDT06, LS18], low-rank games [Meh14], anonymous games [CDO15], and tree games [DFS20].

More relevant to the topic of smoothed analysis, it is known that when equilibria do not fluctuate when the input is perturbed, finding equilibria can be done efficiently [BB17]. Furthermore, a game chosen at random is likely to have easy-to-find equilibria [BVV07].

Spielman and Teng [ST06, Open Question 11] ask whether there is a relation between approximation hardness and smoothed lower bounds: the former implies the latter, but little else is known regarding smoothed lower bounds. For the case of integer linear programs over the unit cube, Beier and Vöcking [BV06] show that a problem has polynomial smoothed complexity if and only if it admits a pseudo-polynomial algorithm. Note that a pseudo-polynomial algorithm can be used to approximate by truncating input numbers. For other problems, we are aware of a few papers that argue smoothed complexity lower bounds via approximation hardness, e.g. [CDT09, HT07, KN07].

2 Preliminaries

We formally define the problem here, and present some remarks. Let be a positive integer. We let be the

th indicator vector. Let

be payoff matrices (corresponding to Alice and Bob). We define a Nash equilibrium to be vectors , called mixed strategies, such that we have that

We say that an equilibrium is -approximate if

For a given equilibrium (often clear from context), we let and be the restrictions of and to , respectively. We let and be the restrictions to , etc. Computing any Nash equilibrium, even -approximate, is known to be -complete [CDT09]:

Theorem 2 ([Cdt09]).

For all , computing an -approximate Nash equilibrium of an bimatrix game with entries bounded in is -complete.

2.1 Remarks on the reduction

The reduction, presented in Section 3, will ultimately take a hard instance of Theorem 2 and transform it into a instance of -SMOOTHED-NASH, for suitable distributions . By the nature of the reduction, if one applies the same reduction with a wider hardness-of-approximation guarantee, one can deduce that for a suitable constant , it is -hard under a randomized reduction to find a -approximate equilibrium of -SMOOTHED-NASH (see, e.g., Eq. 5). This has two interesting implications.

First, this means that if you truncate the output of the distribution

, as well as the uniform distribution sampled in the reduction, to

bits, it is still -hard to find an (approximate) equilibrium for the resulting instance . In particular, the smoothed complexity result is robust to the underlying arithmetic representation of the payoffs.

Second, scaling down the hard instance of Theorem 2 by a small polynomial still maintains an hardness-of-approximation guarantee. Thus, as mentioned in the introduction, the reduction implies that Nash remains intractable even subject to noise (zero-sum + i.i.d.) that is polynomially larger than the worst-case signal.

2.2 Concentration for random bilinear forms

We introduce here the following concentration bound which is useful in our result.

Definition 2 (Subgaussian random variable).

A

-valued random variable

is subgaussian with variance proxy

if for all , . Note that if for some with probability , then is subgaussian with variance proxy .

Lemma 3.

Let be an matrix with independent subgaussian entries with variance proxy at most . For all , with probability at least , all with have

As a corollary, with the same probability, all with have

The proof of this lemma is deferred to the Appendix.

3 The Reduction, and Proof of Theorem 1

First, we show in Section 3.1 the reduction in the case that the noise distribution is symmetric, i.e., the probability of sampling and is identical for all . We then show in Section 3.2 a slight modification which works for any distribution .

3.1 The symmetric case

Let be a sufficiently small constant. Let be any symmetric distribution on . Let be positive integers such that divides , , and is sufficiently large. We divide into blocks which we label We let denote the block length.

Let be payoff matrices. Let denote the all ’s matrix. Let be an matrix whose entries are sampled i.i.d. from the Rademacher distribution (i.e., the uniform distribution on ). Let be an matrix whose entries are sampled i.i.d. from the uniform distribution on . Let be matrices whose entries are i.i.d. sampled from (all distributions independent).121212The to meet the definition of -SMOOTHED-NASH, which specifies that the hard game must have entries between , we can scale the construction (and thus ) by a factor of .

where denotes the matrix, where every entries in block is .

We present here here the final result of this paper. We will refer without proof to a bound on the norm of the equilibrium strategy vectors, and we defer its proof to the rest of the paper, namely Sections 4 and 5. This norm bound is the technical heart of this paper, and the present section illustrates its strength.

We seek to show that equilibria of the reduced game can be used to efficient produce approximate equilibria to the game , which we have assumed is hard to approximate. Let be an equilibrium of . We will show in Section 5 that, with high probability, , even when is a constant. Note that is the dimension of the input game . Define to be distributions over such that for all

Theorem 4.

With probability , we have that , is a -approximate equilibrium of .

Proof.

We claim that is an -approximate equilibrium of with high probability. Assume not, without loss of generality, Alice would benefit from deviating from . That is, there exists such that

(1)

Define to be the uniform probability vector on support , then, the above is equivalent to

(2)

By Lemma 3, we may assume that the concentration inequality holds for , then we know that

(3)
(4)

Combining Eqs. 2,3, and 4 we get

(5)

since . This contradicts that is a Nash equilibrium of .

By a similar argument, Bob does not wish to deviate with high probability. Therefore, is a -approximate Nash equilibrium of . ∎

Since finding a -approximate Nash equilibrium is PPAD-hard [CDT09] when and have constant sized entries, finding the smoothed equilibrium of is PPAD-hard. Since the proofs of Sections 4 and 5 hold when is supported on for constant, this is an instance of -SMOOTHED NASH, and therefore concludes the proof of Theorem 1 when is a symmetric distribution.

3.2 General

Let be any distribution supported on . Let be the distribution on which takes two i.i.d. samples from and subtracts them. Note that is a symmetric distribution, so by the previous section we have that -SMOOTHED NASH is hard. In particular, it is hard to find an equilibrium from the distribution

where and are matrix whose entries are i.i.d. samples from . We can rewrite and , where are all i.i.d. matrix samples from . Thus, the distribution can be rewritten as

This is an instance of -SMOOTHED NASH, and we conclude Theorem 1 for arbitrary , losing a factor 2 on .

4 Equilibria Have Large Support

In this section and the following, we will show the bound on which was required in the proof of Theorem 1. We first show that the support of the equilibria is large with high probability. Then, in Section 5, use this to argue that the weight must be sufficiently spread. The main result of this section is the following lemma.

Lemma 5.

With probability , for every Nash equilibrium of , we have that .

We prove this result using methods partially inspired by [Jon04]. Observe that a Nash equilibrium of requires that

(6)

We seek to show that Eq. 6 cannot hold when the support is sufficiently small.131313In the case of [Jon04], which considers zero-sum games, the LHS of (6) is equal to , so it suffices to bound the probability that the RHS is positive for some and . To do that, we propose a “benchmark” to which both the LHS and the maximum value of the RHS of Eq. 6 are comparable to. To define this benchmark, we begin by introducing a notion of robust partition of the strategy vectors. Consider such that . Let . Let . Let be intervals such that for all and . Let such that

We say that is sparse if it has at most nonzero coordinates; otherwise we say is dense. Let be the sum of the sparse ’s and be the sum of the dense ones. Note that . Now define the following quantity

We call the benchmark for . This quantity will appear in a number of concentration/anti-concentration inequalities. First, we show a key anticoncentration inequality concerning this robust partition.

Lemma 6.

Assume that is the uniform distribution on (i.e., the Rademacher distribution). There exists a universal constant with the following property: For all such that , with probability at least over

The proof of the above lemma is deferred to the Appendix. The following concentration bound will also be of use. For any distribution , we let denote the distribution of matrices with entries i.i.d. samples from .

Claim 7.

Let be any distribution on . There exists a universal constant such that for all , with probability over , for all such that , we have that

Proof.

Apply Lemma 3 to with . Then, there is a universal constant such that with probability , for all with norm ,

Thus, since the entries of have absolute value at most ,

Thus, we can set . ∎

These lemmas will allow us to prove Lemma 5. We present first the following facts about equilibria in random games.

Proposition 8.

With probability , for nonempty there is at most one Nash equilibrium of with and . Further, with probability all such equilibria have .

Proof.

Fix nonempty . Fix . Assume without loss of generality that . Denote as the sub-matrix of restricted to rows indexed by and columns indexed by . For any equilibrium with supports and , we have that for all , when treating and as - and -dimensional vectors, respectively. Therefore,

(7)

Since all the entries of are drawn independently from a continuous distribution, the null space of the linear system (7) has dimension with probability . Since the null space must have dimension exactly . Thus, , which implies that and the solution is unique, as there can be at most one vector in a -dimensional subspace with coordinates summing to one. By a similar argument is also unique.

Since there are only finitely many choices of and , with probability the proposition holds for all Nash equilibria simultaneously. ∎

With probability , all equilibria of and will have the same support size, and further, for every pair of possible supports and there is at most one equilibrium. We let denote the probability distributions of strategies in this equilibrium.

We can now prove Lemma 5

Proof of Lemma 5..

Assume (which happens with probability ) that the event described in Claim 7 occurs for . Fix with . We seek to show that with probability at most , and can be the support of a Nash equilibrium. By Proposition 8, we can assume that .

Also by Proposition 8, with probability , there is at most one equilibrium on the game with full support. Note that and , if they exist, are independent of the entries of and outside of . As mentioned earlier in the section, in order for the equilibrium to extend, the Ineq. 6 must hold:

Say that is -good if . By Lemma 6, we know that with probability at least . Independently, we have that with probability at least (since

is a mean-zero matrix distribution). Therefore, both this event happens with probability at least

.

Likewise, say that is -good if . By the same argument, this also happens with probability at least . Furthermore, the -good events and -good events are independent of each other because each event is based on a disjoint subset of entriesZ from and .

Since and are probability distributions, there exists and such that and . Let be the indices of the blocks such that and . Since we assume that , we have that and both have size at least .

Now, for any good and good , we have

which contradicts Ineq. 6. Thus, there must either be no good or there is no good . This happens with probability at most

where we use in the last inequality that is sufficiently large. The number of pairs with support at most is at most

Note that for sufficiently large, . Thus, all equilibria have support size greater than with probability at least . ∎

5 Equilibria Have Small norm

Towards showing the missing bound in the proof of Theorem 1, the previous section showed that with high probability, any equilibrium must have polynomially large support. We complete here the proof of the norm bound, which in turn completes the proof of Theorem 1.

Lemma 9.

With probability , for every Nash equilibrium of , we have that .

We must, however, begin this section with a few technical results. We will need the following theorem, which is derived from the fact that the VC-dimension of the set of halfspaces in has VC-dimension at most – that is, the VC-dimension of is at most . (See e.g. [Wai19], Example 4.21.)

Theorem 10 (Multivariate Glivenko-Cantelli).

Let be a random vector in and let be independent copies of . For all , with probability ,

We also need the following Littlewood-Offord-type theorem.

Theorem 11 ([Rv15], Theorem 1.2).

Let be real-valued independent random variables with densities almost everywhere bounded by . Let with . Then the density of is bounded by almost everywhere.

The following lemma, which we obtain as a corollary of these two theorems, allows us to argue that the entries of a product of a random matrix with a fixed vector are relatively spread out.

Lemma 12.

Let be positive integers. Let be an -valued random variable with density bounded by . Let be independent random vectors in whose coordinates are independent copies of . With probability , for all unit vectors and all intervals ,

Proof.

By Theorem 10, with probability at least , the CDFs of and the empirical distribution of have distance at most , for all . So it suffices to show that for every unit , . This follows immediately from Theorem 11. ∎

Finally, this lemma allows us to prove the following claim.

Claim 13.

Let be a distribution on whose probability density is at most everywhere. Let . With probability , for every with and , there exists disjoint of size at least each such that for all unit vectors with support in there exists such that

for all
Proof.

For every of size at most , apply Lemma 12 to the rows of restricted to the columns of (so ) with . Thus, with probability , for every unit vector supported on and every interval of length , the number of such that is at most

choices of for which falls in that interval. Since , this implies there exist , and disjoint of size at least such that

for all

Taking the union bound over all choices of we get this all happens with probability at most

We can now prove Lemma 9.

Proof of Lemma 9.

With probability , by Lemma 5, for every equilibrium of with support and , respectively, we have that . Since there are blocks. By the pigeonhole principle there exists such that .

With probability , Claim 13 holds with for both and . Further, with probability at least , Lemma 3 holds for and with .

We seek to show that any large-support equilibrium also has small norm. Assume for sake of contradiction (and without loss of generality) that . Let and be the set of coordinates of which are greater than . Clearly . Let be the coordinates of supported on and be the remaining coordinates. Observe that

(8)
(9)

Applying Claim 13 for and the sets and the vector , there exists and such that (scaling by )

for all