Combinatorial auctions have been a driving force of Algorithmic Game Theory since its inception: how should one allocate goods among interested parties? That is, if a central designer has a setof indivisible goods to allocate, and each of players has a valuation function (private, known only to player ), we wish to partition the items to maximize the social welfare (, where denotes the items received by player in the partition).
This fundamental problem has received significant attention in various models: with or without incentives, with or without restrictions on valuations, with or without computational limits on the players, etc. In this paper we prove standard communication lower bounds when players have subadditive (also called complement-free) valuations.111A valuation function is subadditive if for all . That is, our lower bounds rule out the existence of good mechanisms even when players honestly follow the intended protocol, are computationally unbounded, and are assumed to have subadditive valuation functions. The study of combinatorial auctions specifically through the lens of communication complexity has a rich history dating back to early works of Blumrosen, Nisan, and Segal [Nis00, BN02, NS06], as such lower bounds sidestep challenging debates on appropriate behavioral assumptions. See Section 1.3 for a high-level overview of this literature, and specifically the role of communication complexity.
State of the art.
On this front, the state-of-the-art is fairly remarkable: without any restrictions on the valuations, a -approximation can be achieved in communication [LS05], and this is tight [NS06]. For fractionally subadditive valuations (also called XOS),222A valuation is fractionally subadditive if it can be written as a maximum over additive functions. a -approximation can be achieved in communication [Fei09], and this is tight [DNS10]. For subadditive valuations, a -approximation can be achieved in communication [Fei09], and no better than a -approximation can be achieved in communication [DNS10] (so this is tight as ). As such, remaining open problems in this direction are scarce. The resolution of one such problem is the focus of this paper.
The case of .
While Feige’s -approximation is tight as , the case was posed as an open problem in [DNS10, Fei09]. It initially may seem unusual for the case to be singled out when the asymptotics are resolved, but there is substantial difference in the merits of a -approximation for and . Specifically, Feige’s -approximation for employs an incredibly sophisticated LP rounding, but the same guarantee is achieved by numerous trivial algorithms when : (a) allocate all of to a uniformly random player, (b) allocate each item independently to a uniformly random player, (c) ask each player to report , and award to the highest bidder. Note that (a) and (b) are particularly trivial in that they are completely oblivious to the valuations. (a) and (c) are particularly trivial in that they maintain their guarantee even without subadditivity. All three can trivially be made into truthful auctions ((a) and (b) don’t even solicit input, and are therefore truthful. (c) is simply a second-price auction on the grand bundle ). So resolving the gap between and for is not just a question of determining the optimal constant, but really a question of whether it is possible to achieve any non-trivial guarantee. The main result of this paper answers no:
Main Result (Informal).
For two subadditive players, the aforementioned trivial protocols ensuring a -approximation are optimal among those with subexponential communication.
Before overviewing our construction and extensions, we wish to highlight two immediate implications of our results for combinatorial auctions with strategic bidders, via two recent reductions which renewed further interest specifically in the case.
(1) The power of truthful vs. non-truthful communication-efficient protocols: The central driving theme of algorithmic mechanism design is understanding the relative power of truthful vs. non-truthful “efficient” protocols [NR01]. Remarkably, when “efficient” refers to “communication-efficient,” no separation is known to exist — for any valuation class or agent number — despite significant gaps in the state-of-the-art approximation ratios (see Section 1.3 for further detail, along with a brief discussion of related results concerning computational efficiency). Recent work of [Dob16b] provides a deep structural connection between truthful communication-efficient combinatorial auctions and simultaneous (non-truthful) communication-efficient combinatorial auctions specifically when ,333There are also implications when , but not quite as strong as for . thereby proposing extensive study of the case to search for the first separation. On this front, our result proves that in fact no separation exists for subadditive buyers, as the aforementioned trivial protocols (now proved to be optimal) are also truthful.
(2) Price of Anarchy of simple mechanisms: One measure by which the performance of (non-truthful) combinatorial auctions is quantified in strategic settings is the (Bayesian) price of anarchy (BPoA), defined as the worst ratio between the (expected) welfare of the worst equilibrium and the optimal (expected) welfare. For subadditive valuations, simultaneous first price auctions are known to have BPoA at least [FFGL13], and this is tight, even for two agents [CKST16]. Can auction formats other than first price do better? Roughgarden provides a framework for translating communication lower bounds to BPoA bounds [Rou14]. Together with our new lower-bound result, the framework (in particular Theorem VI.1) immediately implies that no auction format with sub-doubly-exponentially many strategies achieves BPoA better than . This proves that simultaneous first-price auctions are optimal among this class for all .
1.1 Main Result and Intuition
Any (randomized) protocol that guarantees a -approximation to the optimal welfare for two monotone subadditive bidders requires communication .
We now provide some intuition for the main steps. The first step in our proof is the construction of a new class of subadditive functions (Section 3). One key feature of our class, if it is to possibly demonstrate hardness better than , is that it must not also be fractionally-subadditive (due to Feige’s -approximation [Fei09]). Only one general construction exists in prior work (based on Set Cover — see Section 3 for precise description) [Fei09, BR11a].444It is known that every subadditive function admits a fractionally-subadditive function that is -close to [Dob07, BR11a], and this bound is tight by the construction given in [BR11a], based on set cover. It is also known that any valuation function for which for any non-trivial is also subadditive. But such functions trivially admit a -approximation, and therefore don’t serve as a useful starting point. This class is our starting point.
From here, though, we encounter the following barrier: Consider the following line of reasoning, assuming that there exists some set for which . Then allocating player either or uniformly at random (and the rest to player ) guarantees welfare at least by subadditivity of . So if , this allocation guarantees a -approximation. If not, then , and awarding all items to player guarantees a -approximation. It is not hard to combine these observations into a simple, deterministic protocol guaranteeing a -approximation whenever such a set exists.
So in order to possibly demonstrate hardness better than , our class must further have the property that for all , and all in our class, . That is, must essentially appear additive at the large scale (but may be subadditive at smaller scales).
Indeed, our construction starts with the previous Set-Cover construction and essentially hard-codes that for all , in a way that maintains subadditivity. We defer further details to Section 3, but do wish to note that this construction itself is likely to be of independent interest for future work, due to the scarcity of known subadditive functions that are “far from” fractionally-subadditive. Indeed, our results are the first instance where a non-trivial approximation guarantee is achievable for fractionally-subadditive valuations, but no non-trivial approximation is achievable for subadditive.
From here, we are able to show that our constructions are rich enough to encode Equality.555In Equality, Alice and Bob are each given -bit strings as input, and are asked to decide whether they are equal or not. Equality is known to require deterministic communication to solve, but admits efficient randomized protocols. Essentially, the property is extremely convenient, as it immediately implies that whenever . As such, our remaining task is to find a doubly-exponentially-large subset of our class of valuations for which: (a) for all and (b) for all . By the convenient property, beating a -approximation when both players have valuations from this subset is exactly deciding whether , thereby completing a reduction from Equality. Of course, this only proves our claim for deterministic protocols, as Equality is only hard for deterministic communication. We include a complete proof of this reduction in Section 4 in order to highlight the important aspects of our construction without yet requiring any advanced communication complexity.
Finally, we prove our full lower bound for randomized protocols in Section 5. Unfortunately, our construction is really an instance of Equality, and is extremely unlikely to admit a reduction from known problems that require exponential randomized communication (such as Disjointness or Gap-Hamming-Distance). Instead, we propose a new “near-Equality” problem (that we call Exist-Far-Sets), and directly prove that it requires exponential randomized communication via the information complexity approach of [BJKS04, Bra12, BGPW13]. While these tools are now standard in the communication complexity community, they have yet to break into the AGT community. As such, we provide a full exposition in Section 5 (and the associated appendices). Again, we wish to note that Exist-Far-Sets itself may well be of independent interest for future work, especially at the intersection of communication complexity and mechanism design where there is demand for such constructions.
Our main result concerns subadditive valuations (for which we prove that a -approximation is optimal), which are a proper superclass of fractionally-subadditive (for which a -approximation is previously shown to be optimal [Fei09, DNS10]). In Section 6, we further consider the space between fractionally-subadditive and subadditive valuations via the Maximum-over-Positive-Hypergraphs (MPH) hierarchy [FFI15]. We postpone a formal definition to Section 6, but note that fractionally-subadditive valuations are equivalent to MPH-, that all monotone functions lie in MPH-, and that all subadditive functions lie in MPH-. Our second result is a new protocol for welfare-maximization with two MPH- bidders:
There exists a protocol that guarantees a of the optimal welfare for two bidders whose valuations are both subadditive and MPH- with communication.
In particular, our protocol is an oblivious rounding of the configuration LP.666That is, while communication is indeed needed to optimally solve the configuration LP, no further communication is necessary in order to round the resulting solution. See [FFT16] for further discussion on the merits of oblivious versus non-oblivious rounding. We also wish to note an important corollary of this theorem, when combined with our main result. Our main result proves that a -approximation is impossible with subexponential communication. As all subadditive functions are MPH-, this implies that our protocol and lower bound are tight even up to lower-order terms.
Additionally, as our construction does not admit a -approximation in subexponential communication, it establishes the existence of a constant such that our constructions are provably not MPH-. This serves as an additional proof for the existence of subadditive functions that lie in high levels of the MPH hierarchy. The key property we use to claim our guarantee for MPH- may also be useful for future work to claim lower bounds on the MPH level of specific functions.
1.3 Related Work
Communication complexity of combinatorial auctions.
The works most related to ours concern the standard communication complexity of combinatorial auctions. The tables below summarize prior work for various valuation classes. While the table is most relevant for the present paper, the general table is included for reference. Note that no separate row is needed for hardness of truthful communication, because no such results are known (aside from general communication hardness).
|Truthful comm. protocol||[Trivial]||[Trivial]||[Trivial]||[Trivial]|
For context, it is worth noting that all referenced (truthful or not) communication protocols take one of two forms. The first is via solving a particular LP relaxation (called the configuration LP) and rounding the fractional optimum [FV10, Fei09, LS05]. The second is via mechanisms which randomly sample a fraction of bidders to gather statistics, then run a posted-price mechanism on the remaining bidders [Dob16a, Dob07]. Both classes of mechanisms require bidders to communicate demand queries. That is, bidders are asked questions of the form: “For item prices , which set of items maximizes ?” All of the aforementioned protocols/mechanisms make polynomially many demand queries, and have further polynomial-time overhead.
Recent work of [Dob16b] proves a surprising connection between two-player truthful combinatorial auctions, and two-player simultaneous (non-truthful) protocols. In particular, any separation between the approximation guarantees achievable by communication-efficient protocols and communication-efficient simultaneous protocols would constitute the first separatation between truthful and non-truthful communication-efficient protocols. Such separations were already known for large [ANRW15, Ass17], but not for (and therefore aren’t relevant to Dobzinski’s framework). As such, the setting is now receiving extra attention, although the desired separation still remains elusive [BMW18].
Related results on combinatorial auctions.
As previously referenced, combinatorial auctions are studied via other complexity lenses as well. The most popular alternative is the value-queries model, or standard computational complexity. That is, each bidder is capable only of querying their valuation function on a given set (value query), or has access to the explicit (poly-sized) circuit which computes a value query. In both models, a tight -approximation is known for submodular valuations [Von08, MSV08, DV12b], and a tight -approximation is known for XOS and subadditive valuations [DNS10]. To reconcile these latter impossibility results with the above-referenced positive results, observe that it generally requires exp() value queries (or is NP-hard with explicit circuit access) to compute a demand query. Unlike the communication model, strong separations between guarantees of truthful and non-truthful mechanisms are known in these models [PSS08, BDF10, BSS10, DSS15, Dob11, DV12a, DV12b]. It is also worth noting that some of these approaches also yield communication lower bounds for the restricted class of Maximal-in-Range/VCG-based protocols [BSS10, BDF10, DSS15]. For further details of these results, see [DSS15, Table 1].
We study the communication complexity of welfare maximization in two-player combinatorial auctions. Our main result establishes that the trivial -approximations are in fact optimal among all protocols with subexponential communication. We additionally develop a -approximation whenever both buyers are subadditive and MPH-. Our key innovation is a new class of subadditive functions that are “far from” fractionally subadditive, and may be of independent interest for future works. In addition to resolving an open question of [Fei09, DNS10], our results establish the following corollaries: (a) There is no gap between the approximation ratios achievable by truthful and not-necessarily-truthful mechanisms with communication for two subadditive bidders, (b) For any number of subadditive bidders, simultaneous first price auctions achieve the optimal price of anarchy () among all auctions with sub-doubly-exponentially-many strategies (via [Rou14]), (c) Our lower bound is tight even up to lower order terms ( is achievable in communication, but no better).
We consider the following problem. There is a set of items. Alice and Bob each have a valuation function and , respectively that takes as input subsets of and outputs an element of . Moreover, and are both monotone ( for all ) and subadditive (). Alice and Bob wish to communicate as little as possible about their valuation functions in order to find a welfare-maximizing allocation (that is, the maximizing ). Formally, we study the following decision problem – observe that this is a promise problem for which if the input does not satisfy the promise, any output is considered correct.
Definition 2.1 (Welfare-Maximization()).
Welfare-Maximization is a communication problem between Alice and Bob:
Alice’s Input: , a monotone subadditive function over ; and a target .
Bob’s Input: , a monotone subadditive function over ; and a target .
Promise: . Also, there either exists an satisfying , or for all , .
Output: 1 if , ; 0 if , .
We will sometimes drop the parameter when it is irrelevant. We will also refer to any protocol solving as an -approximation for .
Also of interest is the corresponding search problem, which instead asks Alice and Bob to find an maximizing (and an -approximation is a protocol guaranteeing a satisfying ). It is easy to see that any -communication protocol for the search problem implies a -communication protocol for the decision problem (with an extra round of communication). As such, we will prove all lower bounds against the decision problem (as they immediately imply to search as well), and develop all protocols for the search problem (as they immediately imply to decision as well).
3 Main Construction
In this section, we present our base construction. In subsequent sections, we show how to leverage this construction to derive our lower bounds. We begin by considering a collection of subsets where each , and defining a useful property that may possess. Throughout this section, let denote an even integer . Some proofs are deferred to Appendix A.
Definition 3.1 (-sparse).
We say that is -sparse if for all , .
That is, is -sparse if there do not exist elements of such that their union is the entire ground set . We now follow [Fei09, BR11a] in defining a class of valuation functions parameterized by a collection of sets. Specifically, let be an -sparse collection. For , define
where we say “ is covered by ” if . That is, is the smallest number of sets from whose union contains , or some large number if there are no such sets.777Defining if is not covered by would have worked as well. We can now define our valuation function :
If , then define and .
For any whose value is not defined in 1, .
It is not immediately clear that is well-defined; indeed, if and are both , then is doubly defined. Fortunately, this can never occur when is -sparse.
If is -sparse, then is well-defined.
Now, we would like to prove that is monotone and subadditive whenever is -sparse (Corollary 3.3). The following facts about and highlight the key steps in the proof.
Let be -sparse. Then:
is monotone and subadditive.
For all , .
If or , then .
If then .
For all , .
If is -sparse, then is monotone and subadditive.
Functions of the form will form the basis of our lower bound constructions, which we overview in the following sections.
4 Deterministic Protocols for Subadditive Valuations
The construction in Section 3 gets us most of the way towards our deterministic lower bound. The remaining step is a reduction from Equality. To briefly remind the reader, Alice receives input , and Bob receives input . Their goal is to output yes if for all , and no otherwise. It is well-known (see, e.g., [KN97]) that any deterministic protocol for Equality requires communication .
For any even integer , any deterministic communication protocol that guarantees a -approximation to requires communication . In particular, a guarantee of requires communication , and a guarantee of requires communication .
Before proceeding with our construction, we’ll need one more property of collections of sets:
Definition 4.1 ([Ks73]).
A collection is -independent if is -sparse whenever .
In other words, is -independent if we can choose either or independently, for each , and form an -sparse collection no matter our choices. We now proceed with our reduction, which relies on the existence of large -independent collections (such collections are known to exist; at the end of this section we give a precise statement and a proof appears in Appendix A for completeness).
Let be an -independent collection with . Then any deterministic communication protocol that guarantees a -approximation to the optimal welfare for two monotone subadditive bidders requires communication at least .
Let be -independent. For each , define , and . Now, consider an instance of Equality where Alice is given and Bob is given . Alice will create the valuation function , where (i.e. Alice builds by taking either or , depending on ). Bob will create the valuation function , where . Observe first that and are indeed well-defined, monotone, and subadditive as is -independent (and therefore and are both -sparse).
Observe that if , then and moreover . So immediately by part 2 of Lemma 3.2, the maximum possible total welfare is (indeed, any partition of the items gives welfare ). On the other hand, if there exists an such that (without loss of generality say that and ), we claim that welfare is achievable. To see this, consider the allocation which awards to Alice and to Bob. Indeed, (as ), so . Similarly, , so , achieving total welfare .888As an aside, note that welfare exceeding is not possible, as Alice and Bob each value all non-empty sets at least at , and therefore value all strict subsets of at most at by 2. Therefore, the only way Alice or Bob could have value exceeding is to get all of , meaning that the other player receives value .
So assume for contradiction that a deterministic -approximation exists to the optimal welfare for monotone subadditive bidders with communication . Then such a protocol would solve Equality with communication by the reduction above, a contradiction. ∎
Finally, in the next lemma we show how large can be while guaranteeing an -independent collection of size to exist. This suffices to complete the proof of Theorem 4.1. The lemma is based on a known existential construction using the probabilistic method, which we repeat for the sake of completeness in Appendix A (explicit constructions of comparable guarantees also exist [Alo86]).
For all , , and , there exists a -independent collection of subsets of of size .
5 Randomized Protocols for Subadditive Valuations
The construction in Section 4 carries much of the intuition for our randomized lower bound. However, we clearly cannot reduce from Equality and get a randomized lower bound, as Equality admits randomized communication-efficient protocols. As such, we will instead directly show that a certain “near-Equality” problem requires exponential randomized communication. Our proof uses the information complexity approach popularized in [BJKS04, Bra12, BGPW13], which is now standard in that community. In order to introduce these tools to the AGT community, we will provide a complete exposition starting from the basics.
Let’s first be clear about what a randomized protocol looks like. Alice and Bob have access to a public infinite string of perfectly random bits, . All messages sent by (e.g.) Alice may therefore depend on her input, any messages sent by Bob, and
. At the end of the protocol, Alice and Bob will guess yes or no, and the answer should be correct with probability.999As usual, the bound of is arbitrary, as any protocol with success probability can be repeated independently times to achieve a protocol with success probability (and then further repeated to achieve success ). The protocol is only “charged” communication for actual messages sent, and not for randomness used. The main result of this section is as follows:
Theorem 5.1 (Randomized hardness).
Any randomized protocol that guarantees a ()-approximation to requires communication complexity .
Let’s now understand the issue with our previous construction. Aside from the fact that the previous proof clearly does not extend to a randomized lower bound, the construction itself admits a good randomized algorithm. Specifically, let be some -independent set, and let exactly one of be in for all (and also let exactly one of be in ). Let Alice have valuation and Bob have valuation . The problem is that Alice and Bob are still just trying to determine whether or not (that is, if , then the optimal welfare is at most . If not, then the optimal welfare is ). Since and are both subsets of , the randomized algorithm for Equality for inputs of size works.
The natural idea to try next is to reduce from a problem like Disjointness instead (for which randomized protocols indeed require exponential communication). Let’s see one natural attempt from our previous construction and why it fails (just for intuition, we will not exhaustively repeat this for all possible reductions). Again let denote an -independent collection, and again consider any instance of Disjointness of size (recall that and are bitstrings of length and Disjointness asks to decide whether or not there exists an index with ). A first attempt at a reduction might be to let contain for all such that , and contain all such that (but will never contain , and will never contain ). Indeed, with this construction if there exists any index with , the optimal welfare will be (give Alice , and Bob ). Unfortunately, even if there does not exist an index for which , the welfare can still be . To see this, consider any index for which . Then . Moreover, as , and is -independent, (because is -independent, neither nor can be covered with fewer than of the other sets in . As such, ). So while in the “yes” case, the welfare is indeed just like our previous reduction, the welfare in the “no” case will be as opposed to , proving only that a -approximation requires exponential randomized communication (which is already known). Of course this is not a formal claim that no reduction from Disjointness is possible, but provides some intuition for why searching for one (or from Gap-Hamming-Distance, etc.) is likely not the right approach.
The issue is that our construction is getting much of its mileage from the fact that for all , and reducing from any problem except Equality fails to make use of this. So the plan for our new construction is to observe that if and are almost the same (in a precise sense defined shortly), then we can still claim that for all .
The main idea of our construction is as follows: consider still an -independent set . For each , rather than adding either or to , we will add either or , where is a uniformly random element of (and ditto for ). Adding this random element to each set barely changes the welfare, but makes it significantly harder for Alice and Bob to figure out whether their valuations are nearly identical or not. We now proceed with the construction, followed by a complete proof.
Two ordered collections of subsets of are -compatible if
for all .
Either or for all .101010 represents symmetric difference, .
are -sparse, as are .
For any subset of size less than , at least one of contains , as does at least one of .
The main idea is as follows: for any -compatible , consider the valuation functions and . If for some , , this roughly corresponds to the “not equal” case in the previous construction, and welfare near is achievable. If instead, for all , this roughly corresponds to the “equal” case in the previous construction, and welfare near is the best achievable. We first state this formally, and then follow with a proof that randomized protocols require exponential communication to distinguish these two cases.
Let be -compatible. If for some , , then welfare is achievable between and . Otherwise, the maximum achievable welfare is at most .
Suppose that for some , . Consider the allocation that awards to Alice and to Bob. Then Bob clearly has value , as . Also, as and , we necessarily have . This implies that , and therefore . So welfare is achievable (and again, optimal, as no bidder can achieve value without receiving all of ).
Now suppose that for all , , and further suppose for contradiction that total welfare is achievable, by giving to Alice and to Bob. Then one of the players (without loss of generality, say it is Bob) has value , so it must have been the case that was defined to be , and .
Now, observe that because for all that . Indeed, let . Then there is exactly one element in that is not also in , and we therefore conclude that . By criterion 4 of -compatibility, there is some that contains all of these elements, and so , witnessing that .
Finally, if then , so the total welfare is at most , a contradiction. Otherwise, , which we claim implies that . Indeed, if , then it is because . But then we would have , implying a cover of with and contradicting -sparsity of . Observe that in both cases we may conclude that .
Now we may conclude that the total welfare is , again a contradiction. We have now reached a contradiction from all branches starting from the assumption that welfare is achievable, and may now conclude that the maximum possible welfare is indeed , as desired. ∎
The remaining step is now “simply” to show that distinguishing between the two cases requires exponential randomized communication.
5.1 Far-Sets and Exist-Far-Sets
Towards proving our lower bound, we’ll define the following two problems, which may themselves be of independent interest, at least within the combinatorial auctions community, as a “near-Equality” problem which requires exponential randomized communication. Below, note that both Far-Sets and Exist-Far-Sets are promise problems: if the input doesn’t satisfy any of the stated conditions, arbitrary output is considered correct.
Definition 5.2 (Far-Sets()).
Far-Sets is a communication problem between Alice and Bob:
Alice’s Input: , with .
Bob’s Input: , with .
Promise: Either or .
Output: if ; 1 if .
Definition 5.3 (Exist-Far-Sets()).
Exist-Far-Sets is a communication problem between Alice and Bob:
Alice’s Input: . Each .
Bob’s Input: . Each .
Promise: and are -compatible.
Output: . Observe that if the Exist-Far-Sets promise above is satisfied, then by definition the Far-Sets promise is satisfied for all (but not necessarily vice versa — the Exist-Far-Sets promise is strictly stronger due to -sparsity).
Observe that Exist-Far-Sets is exactly the problem referenced in Lemma 5.2 (we state this formally below). Therefore, the goal of this section is to lower bound the randomized communication complexity of Exist-Far-Sets.
Let be such that every randomized communication protocol which solves any given instance of Exist-Far-Sets() with probability at least has communication complexity at least . Then every randomized communication protocol which solves any given instance of with probability at least has communication complexity at least .
Our plan of attack will look very similar to Braverman’s lower bound on the randomized communication complexity of Disjointness [Bra12].
5.2 Information Theory Preliminaries
Here, we provide some basic facts about information theory and information complexity. These are the standard preliminaries one would find in a paper on information complexity (e.g. [BGPW13]). Below, when we refer to a distribution , we use to denote the probability that is sampled from
. All logarithms taken in this section are base-2. Also for this section, all distributions and random variables are supported on a finite set. If for some , we let .
Definition 5.1 (Entropy).
Let be a probability distribution over a finite set
be a probability distribution over a finite set. The (Shannon) entropy of , denoted by , is defined as . If is a random variable distributed according to , we also write .
Definition 5.2 (Conditional Entropy).
Let and be two random variables supported on a finite set . Then the conditional entropy of , conditioned on is .111111To be clear, by we mean the entropy of the random variable , when drawn conditioned on the event .
Observe that as is a strictly concave function, for all (with equality iff and are independent).
. Here, denotes the entropy of the random variable .
Fact 5.4 above intuitively says that the entropy of a tuple of random variables is equal to the entropy of the first, plus the entropy of the second conditioned on the first. Note that if and are independent, then the joint entropy .
Definition 5.3 (Mutual Information).
For two random variables , the mutual information between and , denoted by is: .
Definition 5.4 (Conditional Mutual Information).
For three random variables , the mutual information between and , conditioned on is denoted by , and .
Fact 5.5 (Chain Rule).
Let be random variables. Then .
Fact 5.5 above intuitively says that the information learned about from (conditioned on ) can be broken down into the information learned about from (conditioned on ), plus the information learned about from (now conditioned on in addition to ).
Definition 5.5 (KL Divergence).
We denote by the Kullback-Leibler Divergence between
the Kullback-Leibler Divergence betweenand , which is defined as .
For any random variables , . Here, denotes the random variable conditioned on , and denotes the random variable conditioned on .
Fact 5.7 (Pinsker’s Inequality).
For any pair of random variables of finite support, . Here, .
Definition 5.6 (Information Complexity).
The (internal) Information Complexity of a communication protocol with respect to a distribution over pairs of inputs is defined as follows. Let denote the random variable which is the transcript produced when Alice and Bob participate in protocol with inputs , when are drawn from . Then .
Above, the “transcript” refers to all communication between Alice and Bob (including the order bits were sent, who sent them, etc., and including any public randomness) when participating in protocol . In particular, it is always possible to glean the output produced by from viewing the transcript (but possibly additional information). Informally, the Information Complexity captures the amount of information Alice learns about Bob’s input from participating in (given that she already knows her own input, the public randomness, and that their joint input is drawn from ), plus what Bob learns about Alice’s input. Intuitively, it should be impossible for a protocol to convey bits of information without bits of communication. Indeed, this is the case:
Lemma 5.8 ([BR11b]).
For any distribution and protocol , (where denotes the worst-case number of bits communicated during protocol on any input).
We conclude with a few more basic facts about communication protocols. Lemma 5.9 below captures one key difference between communication protocols and algorithms with access to the entire input. Lemma 5.9 below refers to private randomness, which are random bits which are known only to Alice (but not Bob) and vice versa. Such bits are also not counted towards the communication cost of the protocol (unless Alice wishes to share her private randomness with Bob).
Let be a function where denotes the probability that Alice with input and Bob with input produce transcript when participating in a protocol over the randomness of any private randomness used (as public randomness is already accounted for in the transcript). Then there exist functions and such that .
The proof of Lemma 5.9 is straightforward. Essentially, is the probability that Alice doesn’t deviate from transcript with input , conditioned on Bob communicating according to transcript so far. Similarly, denotes the probability that Bob doesn’t deviate from transcript with input , conditioned on Alice communicating according to transcript so far. These probabilities are well-defined because Alice must choose her future messages based only on the transcript so far (including the public randomness) and her input (and Bob’s must be only on the messages so far and ), as well as her private randomness. Once confirming that these probabilities are well-defined, it is easy to see that indeed .
Finally, the lemma below states that lower bounds on the information complexity of any protocol that only uses private randomness also lower bound the information complexity of any protocol which uses public randomness. This initially may seem counterintuitive, since the opposite is true for communication. Both simple claims below have “approximate” versions in the other directions (discussed in the cited references), but we only use the easy directions.
Let be a protocol, and be a distribution over inputs. Then:
If uses private randomness, there exists a protocol using public randomness with exactly the same output as , and . But maybe .
If uses public randomness, there exists a protocol using private randomness with exactly the same output as , and . But maybe .
Both claims in Lemma 5.10
follow by simple reductions. For the first bullet, simply use all odd bits of the public randomness string as private randomness for Alice, and all even bits of the public randomness string as private randomness for Bob. Then the output of the protocol is identical, and the communication has not changed. However, Bob nowknows Alice’s private randomness, so the protcol may reveal significantly more information than previously (one example to have in mind is that perhaps the protocol has Alice output one uniformly random bit of her input. With private randomness, Bob learns very little about Alice’s input upon seeing the bit. With public randomness, Bob learns exactly one bit of Alice’s input). For the second bullet, simply use Alice’s private randomness as the public randomness. That is, whenever the protocol requests random bits, Alice outputs these bits from her private random string. These bits are completely independent of her input, and therefore reveal no additional information. However, the communication might become enormous, as the randomness is now being directly communicated, and counts towards the communication cost. To use Lemma 5.10, our lower bounds proceed by first lower bounding the information complexity of Exist-Far-Sets with private randomness, using Lemma 5.10 to lower bound the information complexity of Exist-Far-Sets with public randomness, using Lemma 5.8 to lower bound the communication complexity of Exist-Far-Sets with public randomness (if desired, we could then use Lemma 5.10 to further lower bound the communication complexity of Exist-Far-Sets with private randomness). The point is just that exponential communication is required with either public or private randomness.
5.3 From Exist-Far-Sets to Far-Sets
In this section, we show how to lower bound the randomized communication complexity of Exist-Far-Sets, provided we have any lower bound on the information complexity of (certain instances of) Far-Sets. This section is analogous to Section 7.2 of [Bra12], but we repeat the approach here in order to properly introduce these ideas to the AGT community.121212Also, because of the additional promise of Exist-Far-Sets, we are unfortunately unaware of prior work which we can cite as a black-box. But the proof below really follows from exactly the same ideas as [Bra12] after accounting for the promise. To get started, we need some additional notation for promise problems.
Definition 5.7 (Promise problem).
Let be some function mapping , and let . Then the communication problem solving under promise refers to the communication problem which requires Alice with input and Bob with input to output whenever (and they may provide arbitrary output otherwise).
Definition 5.8 (-compatible inputs).
Say that an input is -compatible with respect to and