Submodular Stochastic Probing with Prices

10/03/2018 ∙ by Ben Chugg, et al. ∙ University of Oxford 0

We introduce Stochastic Probing with Prices (SPP), a variant of the Stochastic Probing (SP) model in which we must pay a price to probe an element. A SPP problem involves two set systems (N,I_in) and (N,I_out) where each e∈ N is active with probability p_e. To discover whether e is active, it must be probed by paying the price Δ_e. If it is probed and active, then it is irrevocably added to the solution. Moreover, at all times, the set of probed elements must lie in I_out, and the solution (the set of probed and active elements) must lie in I_in. The goal is to maximize a set function f minus the cost of the probes. We show this problem can be approximately solved via multilinear relaxation and contention resolution schemes. If I_in and I_out admit (b,c_in) and (b,c_out) contention resolution schemes, we give a {c_out c_in,c_out+c_in-1}α(b)-approximation to online SPP, where α(b)=1-e^-b if f is monotone, and be^-b otherwise. These results apply in the online version of the problem: The elements are presented in an arbitrary (and potentially adversarial) order. In the special case when I_in and I_out are the intersection of k and ℓ matroids, we demonstrate that the optimal value for b is 1/2(z+2-√(z(z+4))) when f is monotone, and z+1-W(ze^z+1) when f is non-monotone, where z=k+ℓ and W is the lambert W function. Our results also provide state-of-the-art approximations for online SP when f is monotone, and provide the first approximation to online, adversarial SP when f is non-monotone. Finally, we demonstrate that when f is modular there exists an α-approximation to SP only if there exists an α-approximation to SPP.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Life is hard: There is lots of uncertainty, it is rarely the case that one can gather all the information before making a decision, and information isn’t free. Whether we’re buying a plane ticket for a conference, or a pair of socks for a significant other111We leave the question of whether a pair of socks is the optimal gift for another paper., it is impossible to be aware of every item in the market and gathering the information to make a good decision costs time and resources. The job of computer scientists is to try and model such scenarios. In this paper, we propose the Stochastic Probing with Prices (SPP) model, which combines two previous models—Stochastic Probing [15] and the Price of Information [19]—in order to study decision making under uncertainty where ascertaining new information has a cost.

1.1 Problem Formulation

Let be a set of elements, and , be two downward-closed set systems222 is downward-closed if and implies .. Every element is active with probability , independently of all other elements. In order to determine whether is active, we can probe . However, if is probed and is active, then it must be added to our current solution (which thus consists of all previously probed which are also active). Moreover, at all times the set of probed elements must lie in and the solution must lie in . Thus, if is the set probed elements so far and is the solution, we can only query an element that satisfies and . The set of active elements is determined a priori, i.e., before the execution of the algorithm. Given an instance of the problem, let refer to the set of active elements. We suppose we are given two set-functions, , where is a utility or objective function, and cost is (naturally) a cost or payment function. The goal is to query the elements in such a way as to maximize while minimizing the cost paid for the queries: . In this paper we assume that the cost function is linear: for each there exists a price drawn from some distribution , and . For simplicity, we will assume that is deterministic (i.e., is a point mass). Otherwise, since cost is linear, we can simply replace with to obtain our results for more general distributions. We call this model Stochastic Probing with Prices (SPP). If we can choose the order in which the data is revealed to us is, the problem is offline, otherwise it is online. We draw a distinction between two kinds of online settings: an adversarial setting, in which the order may be chosen by an adversary, and a random-order setting, in which the elements are chosen uniformly at random. In this paper, all of our results pertaining to the online SPP apply in the adversarial online setting. Surprisingly, when is submodular, our algorithms do not seem to yield better approximations in the offline or random-order settings than in the adversarial setting.

SPP is a generalization of the Stochastic Probing (SP) model of Gupta and Nagarajan [15] and borrows the pricing idea from the recently developed Price of Information (PoI) model of Singla [19]—see Sections 1.2 and 1.3. It falls under the general category of what we might call a query-and-commit model of computation, in which irrevocable decisions must be made on the fly. Stochastic matching, packing and the secretary problem are examples of problems studied under such a model [11, 16, 9, 7]. Since SPP is a generalization of SP (as we will see), it is useful in the same application domains as the latter model. However, we believe few problems encountered in practice do not have a cost associated with obtaining information (even if it is only a time cost).

We now formally introduce the models upon which SPP is built, and relevant known results.

1.2 Stochastic Probing

The SP model is precisely the SPP model without the cost function. That is, the goal is to maximize only. It has been a valuable abstraction of stochastic matching and packing, and has been useful in modeling such practical problems as kidney exchange, online dating, and posted price mechanisms [15, 2, 1].

In order the present many of the known results for SP, we need to introduce contention resolution schemes (CRSs), first introduced by Chekuri et al. [6]. Suppose we are given a fractional solution to an LP-relaxation of some constrained maximization problem. The goal is now to round to an integral solution which respects the constraints, without losing too much of the value given by the fractional solution. Intuitively, a -CRS gives a guarantee that if then will be in the solution with probability at least . Thus, if is the linear (or concave) objective function, then , with high probability (w.h.p.). A -CRS gives the same guarantee assuming that , where is a polytope relaxation of the constraints of the problem. (Thus, a -CRS is a -CRS.) Feldman et al. [14] recently extended the notion of CRSs to online settings. We will provide the formal definition of both offline and online CRSs in Section 2.2.

As mentioned above, the general framework of offline SP problems was first introduced by [15] who restrict their attention to modular objective functions. When and admit offline and schemes, they give a -approximation to modular SP. This implies a -approximation when and are and systems [15]. If and are the intersection of and matroids, respectively, then they give a -approximation. Under these same constraints, Adamczyk et al. [2] improve this approximation to via a randomized rounding technique, and give a -approximation in the more general case when is monotone submodular. Online SP was first introduced and studied by Feldman et al. [14]. When and admit and online CRSs respectively, they give a approximation to online modular SP, and a -approximation to online, submodular SP for monotone objective functions. These results hold in the adversarial setting. More recently, Adamczyk and Włodarcyzk [3] initiated the study of random order CRSs, and use such schemes to achieve the first approximations to non-monotone submodular SP in the random-order setting, giving a -approximation.

1.3 Price of Information

Perhaps the first, well-known problem to incorporate costs for querying elements is Weitzman’s Pandora’s Box [21]. Generalizing this problem, Singla [19] recently developed the PoI model.

In this setting, we are given a single set system333Although it is possible to generalize the model, as discussed in [19] and each element has a value which is drawn from some distribution

. The distribution is not limited to a Bernoulli distribution as in the SP model. Each element

also a price, . The goal is to query a set of elements in order to maximize

where is some function that depends only on the chosen set, not the values of the elements. The function is called a semiadditive function for this reason. Besides the fact that one must pay for making queries in this model, the key distinction is that one gets to choose the maximizing argument . That is, this model is not a query-and-commit model. The major result of Singla [19] was to prove that if there was an -approximation to a given problem using what he calls a “frugal” algorithm (essentially, but slightly more general than, a greedy algorithm) when all prices are zero, then there was an -approximation to the problem with arbitrary prices. Intriguingly, in our work we will obtain a similar result for modular SPP—this will be described in detail later.

While SPP is a generalization of SP, there is no strict inclusion relation between SPP and the PoI model. The PoI model is not a query-and-commit model and as such it does not capture SPP. Furthermore, it is unclear how one would expand a PoI problem to use more general classes of objective functions—submodular functions, for example—while this is easily modeled in SPP. Conversely, SPP does not capture PoI because it is more limited in its assumption that an element is either active or inactive, and cannot model more complicated distributions.

In his Ph.D. thesis, Singla does examine the PoI setting with commitment constraints [18]

. In this scenario, the values of the elements change in time as a Markov process, and at each step, we can decide to advance the Markov chain and pay a cost, or stop it and obtain its current value. The setting of activation probabilities can be viewed as a two-stage Markov chain; therefore this model can be viewed as a generalization in that sense. However, there is no outer-constraint, i.e., every element can be queried.

1.4 Our Results

In this paper, we are concerned with two classes of objective functions: modular and submodular. We call the corresponding problems modular and submodular SPP. Note that if is modular then there exists weights such that . Consequently, when all prices are zero, this problem has also been called weighted stochastic probing.

We begin by studying modular SPP. We demonstrate that if there exists an -approximation for SP, then there exists an -approximation for SPP. This result applies in all SPP settings (e.g., adversarial, random order, offline). If there exists an -approximation to modular SP then there exists an -approximation to modular SPP.

For submodular SPP, we demonstrate that slight variations of the algorithm for modular SP of Gupta and Naragajan [15] (using offline CRSs) and the algorithm of Feldman et al. [14] (using online CRSs) give a good approximation to adversarial submodular SPP if and admit and CRSs. Interestingly, our variation of Gupta and Naragajan’s algorithm increases its scope in two significant ways: we do not require the inner CRS to be ordered, nor the ability to choose the order in which the elements are presented. Thus, our version of their algorithm (which is called Offline-Rounding because it uses offline CRSs—Section 4.2) applies to online SPP. Suppose and admit and CRSs and let . There exists a -approximation to adversarial, submodular SPP where if the objective function is monotone, and otherwise.

In the special case when and are the intersection of and matroids, we demonstrate that the optimal value of is when is non-monotone, and when is monotone, where is the Lambert W function [8].444 is defined by . It is also called the product log function. This gives a -approximation in the monotone case, where , and something significantly uglier in the non-monotone case. This calculations are performed in Section 4.3.

A word now on the relationship between our results (which, since SPP is a generalization of SP provide approximation guarantees for SP) and previous results for SP when and are the intersection of matroids. In the offline case, the approximation ratio of Adamczyk et al. [2] strictly beats our approximation ratio. However, in the online version of the problem, our results outperform those of Feldman et al. [14] (who study the adversarial setting) and Adamczyk and Włodarcyzk [3] (who study the random-order setting). This is also shown in Section 4.3. Furthermore, we provide the first results for a non-monotone objective function in the adversarial setting.

We believe that a significant benefit of our results is their simplicity, both with respect to the algorithms and the analysis. As mentioned above, the algorithms are variations on existing algorithms; we hope that this helps provide a more unified view of SP and SPP problems. The analyses use existing techniques which should be familiar to those working in submodular maximization which we hope makes the results easily accessible.

1.5 Proof Techniques

For a modular objective function, there is little daylight between SP and SPP. The former is reduced to the latter by modifying the weights as in Theorem 1.4. No such transformation can be made in the more general case of a submodular objective function however, which thus increases the difficulty of the problem.

Similar to the algorithms presented in [15, 2, 3, 14] for SP, our algorithms proceed by obtaining a fractional to a suitably chosen LP-relaxation of the problem, and then rounding according to both CRSs. The first major difficulty is that while is still a submodular function, it may not be non-negative. Since traditional algorithms for submodular maximization require that the objective function be non-negative, this is a hurdle which needs to be overcome. To do so we use the recent technique of Sviridenko et al. [20] in order to maximize a function of the form , where is concave, is a constant, and is linear. Lemma 4 demonstrates that this objective function (subject to the appropriate constraints) provides an upper bound on the value achieved by the optimal policy. The guarantee is given by Theorem 4.1. The parameter here is crucial: After rounding (in an online fashion), the approximation bound on and may be different. An appropriately chosen value of thus allows us to obtain the approximation factor.

2 Preliminaries

Let

. Given a vector

, let be a random set where each is selected with probability . We will sometimes write to refer to choosing a random set . We call the activation probability of . We emphasize that it is independent of the activation probabilities of other elements. For a set and element , we will often write in place of the more cumbersome . Given a (stochastic) probing algorithm Alg, we will abuse notation somewhat and write to mean the expected value of the solution obtained by querying according to , i.e.,

. Note that the expectation is over the joint distribution

and any randomness in the choices made by the algorithm.

2.1 Modular and Submodular Functions

A function is submodular if for all If this inequality holds with equality, then is modular. Given , write for . We will consider two extensions of a set function to . The first is the multilinear extension [4, 5]:

The second is the concave closure of [6]:

As indicated by the wording is concave. It is well known that . Indeed,

can be viewed as selecting the probability distribution over

maximizing subject to the constraint that , while is the value of under a particular such distribution.

2.2 Contention Resolution Schemes

Let be downward-closed. The polytope relaxation of is the set defined as the convex hull of all characteristic vectors of .

For the rest of this section fix a downward closed set system , and let be its convex relaxation. We now introduce the formal definitions of offline and online CRSs.

[Offline CRS [6]] For , a offline Contention Resolution scheme for is family of (possibly randomized) functions such that for all and : (1) ; (2) with probability 1; and (3) for all , .

The CRS is monotone if for all , . We emphasize that in condition (3) of Definition 2.2, the probability is over both the random set and the CRS, whereas in the definition of monotonicity of a CRS, the probability is taken only over the (possibly) random choices of scheme itself.

[Online CRS [14]] For , a online contention resolution scheme of is a procedure which defines for any a family such that for all , .

Given an online CRS of the authors of [14] define a related offline CRS, called the characteristic CRS . They verify that the characteristic CRS meets the conditions of an offline CRS and demonstrate that the characteristic CRS of a online CRS is a monotone, CRS. If we refer to a CRS without specifying whether it is online or offline, the discussion should apply to both. Unless otherwise stated, we will assume that any CRS discussed in this paper is efficient: that it can be computed in polynomial time.

2.3 A general probing strategy

A permutation or ordering on a subset is a bijection . To say traverse in the order of should be taken to mean iterate over in the order . The following probing procedure will be repeated often enough throughout the paper to warrant a name.

on . Let and . For all in the order if and , then

  1. if then probe and set . If was active, set .

  2. if , then add to with probability .

Return .

We observe that this strategy does in fact produce a valid set of probed elements and a valid solution.

Observation 1.

Let and be the sets returned by GreedyProbing. Let be the set of elements queried by GreedyProbing which were active. Then . Moreover, hence and .

3 Modular SPP

The main result of this section is to verify the intuitive result that when the objective function, , is modular, we can obtain the same approximation ratio to the optimal as in SP (i.e., with no prices). Recall that if is modular, there exist weights such that . Here, we will not draw a distinction between online and offline SPP. This is because the results apply in both settings. We will reduce an instance of SPP to an instance of SP. The result requires only a change of variables, and thus only knowledge of the weights, prices, and activation probabilities in the SPP instance. Thus, if the algorithm for SP is online or applies in any other setting555For example, stochastic probing with deadlines has been examined [15, 14]., so too does the corresponding SPP algorithm.

We begin with an observation that no algorithm worth its salt will query elements whose expected weight is upper bounded by their price.

If is a query strategy which queries where , there exists a query strategy which does not query such that .

Proof.

Given , define to be the strategy obtained by mimicking but refraining from querying any element such that . It follows that is still a valid strategy because and are downward-closed. Moreover, any element queried by which is not queried by adds a non-positive expected value to the solution. ∎

Henceforth we will apply Lemma 3 and assume that for all elements , . In this section we will allow prices to be negative; therefore, it is not necessarily the case that . This section requires that we make comparisons between solutions of SSP and SP, and hence need to introduce the relevant notation. For a probing strategy , let denote the expected value in the SP setting with weights and probabilities . Similarly, let denote the expected value of the solution in the SSP setting with weights , probabilities and prices .

Given an instance of SSP, we define new weights and activation probabilities as follows. For all , let if and otherwise and let for all with , and for with .

For any querying strategy and any weights , activation probabilities and prices , .

Proof.

Let

be the random variable denoting the set of active elements according to the probabilities

. Note that for all , hence . Unwinding definitions now gives

Proof.

Let Alg be an algorithm for linear SP which obtains an -approximation. Let be an instance of linear SPP. Running Alg on weights and probabilities and applying Lemma 3 gives

4 Submodular SPP

We now proceed to the more general problem of submodular SPP. We assume in this section that the prices are non-negative. We will employ the common approach of solving a relaxed linear program (Section

4.1), and then rounding the solution to obtain a probing policy (Section 4.2). First we observe a natural upper bound on the value of the optimal solution, against which we can gauge the quality of approximations. It will be notationally convenient to work with a single polytope instead of both and . Accordingly, we will henceforth let refer to the polytope

Let be the (multi)linear extension of cost. Thus . A natural relaxation submodular SPP is the following program:

(LP)

The following lemma demonstrates that this program is indeed a proper relaxation of our problem.

Let Alg be a (stochastic) probing strategy and let be an optimal solution to (LP). Then .

Proof.

Given Alg, define and by and . It is immediate that and if Alg is a valid strategy. First, we claim that is feasible solution for which it suffices to show that . Recall that by the constraints of the problem, if an element is queried, then its addition to the current set of probed elements and to the solution must be allowed by the constraints of and respectively. Thus, , where we’ve used the fact that whether a particular element is active or not is fixed a priori. Now, notice that where is a particular distribution such that . Conversely, is the maximum over all such distributions, i.e., . Therefore, and so,

4.1 Obtaining a fractional solution

While the problem of maximizing a (non-monotone) submodular function subject to various constraints has been the subject of intense study (e.g., [6, 13, 17, 10]), less is known about combinations of submodular functions. In our case, the difficulty in solving (LP) efficiently arises because the function is not necessarily non-negative. Removing the non-negativity condition in a non-monotone submodular maximization problem makes the problem intractable in general, since, as noted in [6], it may take an exponential number of queries to determine whether the optimum is greater than zero. We must therefore take advantage of the special form of our problem; namely the fact that is linear.

Recently, Sviridenko et al. gave an approximation algorithm for maximizing the sum of a non-negative, normalized, monotone, submodular function and a linear function over a matroid constraint [20, Theorem 3.1]. More precisely, given a submodular function , a linear function and a matroid , with high probability they obtain a set such that minus an arbitrarily small constant term, for any base . The idea is elegant and straightforward, and involves using the traditional continuous greedy algorithm but over the polytope (rather than simply ) where is a guess for the value of . Intuitively, this guarantees that the fractional solution satisfies (where is the linear extension of ). Somewhat surprisingly, restricting the polytope in this way does not damage the approximation to .

In this section we demonstrate that their approach applies to non-monotone submodular functions and arbitrary constraints. The result is summarized as Theorem 4.1. We say a function can be efficiently estimated with high probability if for any , can be determined to within exponentially small error with a polynomial number of queries. For example, if we are given oracle access to a submodular function

, then its multilinear extension can be efficiently estimated with high probability (see, e.g.,

[6]).

Let be a normalized submodular function with multilinear extension which can be efficiently estimated with high probability. Let be a non-decreasing modular function with (multi)linear extension . For any , , , and downward-closed system , there exists a polynomial time algorithm which produces a point such that

with high probability, where

and if is monotone, and otherwise.

Due to the similarity of Theorem 4.1 to Theorem 3.1 in [20] and of its proof to that of the Measured Continuous Greedy algorithm in [13]

, its proof is moved to the appendix. It may be worth taking a moment to discuss the constant

in Theorem 4.1. While it appears to be a free parameter, note that the choice of also affects the value of , i.e., is a function of and . Later, when applying Theorem 4.1, we will choose suitably so that the approximation factors on the terms and cost match (Section 4.3).

4.2 Obtaining a policy

While Theorem 4.1 guarantees the existence a fractional solution which gives a good approximation to the value of the optimal policy, it does not tell us how to query the points. This section focuses on extracting a probing policy from this fractional solution. We give two rounding techniques, which apply depending on what kind of CRSs to which one has access (i.e., online or offline). Let .

Offline-Rounding. Let and be offline CRSs for and respectively. Draw and compute . Run on where is any ordering.

Online-Rounding. Let and be the (random) subsets given by the inner and outer CRSs. If is not online, take . Draw . Run on if is online, and on otherwise where is any ordering (even adaptively and adversarially chosen).

Analysis. For either rounding technique, let , , and be as in Observation 1. Given any run of the algorithm, let be defined as those elements which were either probed and active, or else had negative marginals but were nonetheless added to . Note that is distributed as and thus as . However, it might not be the case that because not all elements in were actually probed. Note that , and are actually functions of and , and it will oftentimes be helpful to write them as such (i.e., ). Finally, we define , where and were the inner and outer CRSs used in the rounding technique (the given schemes in the case of offline rounding, and the characteristic schemes in the case of online rounding). The following lemma uses properties of the CRSs to obtain the inequality which is crucial to the main result.

Let and be monotone and CRSs respectively. Let and set . Then . Moreover, where is the set of elements probed by the algorithm.

Proof.

When conditioning on the choice of the random set , the randomness in and stems purely from the CRSs themselves. Hence, the events and are independent. Additionally, notice that given and , the quantity is monotonically decreasing in . Combining these two facts and conditining on gives

(1)

using the FKG inequality. This implies that . To obtain the other bound, we compute

Rearranging gives the desired result. The final statement follows from noticing that in either rounding scheme, and . ∎

Instead of reasoning directly about the set of elements in the solution, we will reason about the set . The following two lemmas prove useful properties about this set.

Given and with , .

Proof.

The randomness comes only from the CRSs, since and are given. Therefore, . ∎

For any and , .

Proof.

Let . We first observe that regardless of the rounding technique, if advances to having its marginal considered in GreedyProbing then, by definition of , it will be added to . Therefore, it remains only to show that meets the condition and in GreedyProbing, where and are the respective intermediary solutions immediately before is considered by the algorithm. In the case of offline rounding, this is immediate since GreedyProbing is run on which is in and (since and w.p. 1). Now consider online rounding with two online schemes. Here, recall that . Taking we see that . Similarly, . The argument for online rounding when is offline is similar. ∎

This final technical lemma derives a lower bound on the marginal of our solution with respect to that of the optimal’s. The main approximation guarantee will then result from decomposing the objective function into the sum of its marginals and applying the following lemma.

Let and let be any ordering on . For every , where .

Proof.

Fix and let . We have:

where the third equality follows from the fact that if then , and the final inequality follows from submodularity. Let . Now, condition on and write

where the final inequality follows from Lemma 4.2. Both and are decreasing functions of (the former from submodularity and the latter from Lemma 4.2). Therefore, by the FKG inequality, the above is at least

by Lemma 4.2, where the final equality uses the fact that the events and are independent. Combining everything and keeping in mind that is distributed as , we obtain

as desired. ∎

4.3 Approximation Guarantees

Given the rounding policies presented in the previous section, we are now ready to prove Theorem 1.4, and explore the guarantees given in the case where and