Consider a scenario where a decision maker observes a sequence of non-negative real-valued awards, . When an algorithm, denoted by , serving on behalf of the decision maker reaches the award, , it needs to make an immediate and irrevocable decision whether or not to accept the award. If it accepts , the game terminates with an award of ; otherwise, it continues to the next round, and the award is lost forever. The performance of an algorithm for such an online scenario is often measured by the competitive ratio, defined as the worst case ratio between the award accepted by and the maximal award in hindsight.
Without imposing additional assumptions on the input, the competitive analysis framework produces no insights about the design of algorithms for such scenarios. In two natural frameworks, however, known as the prophet and the secretary, good guarantees can be given.
Prophet inequalities refer to guarantees on the competitive ratio in a setting, where every award is drawn independently from a known distribution
. The competitive ratio is the ratio between the expected performance of the algorithm and the expected maximal award, where the expectation is taken over the product distribution, and possibly also over the random coin tosses of , if it is randomized.
Krengel, Sucheston, and Garling (1977; 1978) demonstrated the first fundamental result in this framework, proving the existence of an algorithm with a competitive ratio at least a half. In other words, a “prophet” who observes the entire sequence of realized awards from the outset, and simply takes the maximal award when reached, can gain at most twice the value that a gambler, who makes immediate and irrevocable decisions based on present and past observations only, can gain. Samuel-Cahn (1984) later showed that this competitive ratio can be obtained with a simple threshold algorithm: setting some threshold and accepting the first award that exceeds it.
In the secretary framework, the awards are chosen by an adversary, but the arrival order is assumed to be randomly and uniformly distributed. The performance ofis taken in expectation over the random arrival order, and possibly also over the random coin tosses of , if it is randomized. This performance is measured with respect to the maximal award in the sequence. This framework appeared first in Martin Gardner’s Scientific American column (Gardner, 1966). Gilbert and Mosteller (1966) presented an algorithm that achieved an asymptotic competitive ratio of
for this problem, that is, an online algorithm that picks the maximal award with a probability at least. This bound is asymptotically tight (Note that since the values are chosen by an adversary, bounding the probability of picking the maximal element is equivalent to bounding the ratio between the picked element and the maximal element).
The two frameworks have significant implications for the design and analysis of auction and posted price mechanisms. In recent years there has been a surge of interest in applying results from secretary and prophet scenarios to mechanism design settings. For example, a direct implication of the classical prophet inequality is that a seller who knows the distributions from which buyers’ values are drawn can achieve half of the optimal welfare by posting a single price and selling to the first buyer whose value exceeds the price. Similarly, a direct implication of the classical secretary setting is an auction setting that obtains of the optimal welfare: sample at random a fraction of the buyers and query their value, then set a price at the maximal sampled value and offer this price to the buyers in a random order. Although welfare implications are more direct, these results also lead to strong approximation guarantees on revenue, mainly in single parameter settings, but also in multi-parameter settings, such as matching markets (Chawla et al., 2010; Alaei et al., 2015; Chawla et al., 2007; Kleinberg and Weinberg, 2012; Azar et al., 2014).
Following these observations, a series of works have looked at generalizations of prophet and secretary. The common scenario studied in these works is the following: a collection of feasible sets of elements is given, the values of the elements are revealed one by one in an online fashion, the decision maker makes an immediate and irrevocable decision whether to accept or reject each element (under the constraint that the set of selected elements must belong to at all times), and the value of an accepted set is the sum of values of elements in the set. An example of a feasibility constraint is a cardinality constraint, where one can accept at most elements. This corresponds to selling identical items. It was shown that prophet and secretary with cardinality admit competitive ratios of and , respectively (Kleinberg, 2005; Hajiaghayi et al., 2007; Alaei, 2014).
Additional feasibility constraints include independent sets in matroids (Kleinberg and Weinberg, 2012; Babaioff et al., 2007; Babaioff et al., 2008), polymatroids (Dütting and Kleinberg, 2015), knapsack constraints (Feldman et al., 2015; Duetting et al., 2017), and general downward-closed (Rubinstein, 2016). Recently, Duetting et al. (2017) established a new framework that uses prophet inequalities reasoning to establish pricing mechanisms with good welfare guarantees in combinatorial settings.
Online scenarios with limited returns. All previous works considered scenarios in which one makes immediate and irrevocable selection decisions. In many scenarios of interest, however, decisions are not fully irrevocable. In this work we provide a modeling framework for the study of online scenarios with returns, parameterized by the number of possible returns. We quantify the improvement in the performance of secretary and prophet algorithms as a function of this parameter.
In particular, we consider settings, termed -out-of-, in which the decision maker derives value from elements (e.g., selling identical items), but during the online process, he can select elements. In other words, as elements arrive in an online fashion, the decision maker selects up to elements, but performance is measured by the top elements in the selected set. This setting is mathematically equivalent to a setting in which the decision maker holds at most elements at all times, but can make up to returns as the values of the elements are revealed.
Different variants of prophet and secretary problems from the literature are special cases of this model. For example, the classical secretary and prophet problems correspond to the special case where . Similarly, the secretary and prophet problems with cardinality constraints correspond to the special case where .
As the parameter varies, the model moves gradually from an offline to an online setting. The pure offline setting is the special case , where the decision maker chooses the highest values after observing the entire sequence. The pure online setting is the special case , where no returns can be made. Thus, our study measures the change in the performance of the algorithm as the setting moves gradually from online to offline.
Scenarios of limited returns arise in various settings, including hiring, mechanism design, and job scheduling. For example, sellers may have the option to commit to selling more items than they actually have, taking into account a fine they may need to pay if they fail to deliver. Although the opportunity of returns comes with clear benefits, it also bears some costs, which are known to the designer of each setting. Our results help a decision maker to quantify the benefits due to returns, providing her better tools to weigh the cost of returns against their benefits. We proceed with several examples of economic scenarios exhibiting limited returns.
An employer who interviews candidates for a job that requires employees may be able to tentatively accept a slightly larger number of employees, then choose the best among them. Extra hires, however, may be costly because of regulation and reputation effects.
An air carrier that has a capacity of seats for a given flight may wish to overbook, that is, sell more tickets than the capacity of the plane. In the end, the company can use different methods to sell the tickets to the passengers with the highest value, e.g., offer a monetary compensation to agents who agree to postpone their flight.
Preemption in job scheduling
A scheduler who needs to choose jobs to process may preempt existing jobs as new ones (perhaps more urgent) arrive. Preemption, however, may incur some cost related to maintaining the state of the system or the database.
In this work we study prophet and secretary settings with limited returns under cardinality constraints. Prophet and secretary settings with some level of revocability is relevant under other feasibility constraints as well. For example, in matroid prophet and secretary problems (Kleinberg and Weinberg, 2012; Babaioff et al., 2007; Babaioff et al., 2008), a decision maker selects elements online, so that the selected set is an independent set of the matroid at all times, and the performance is measured by the sum of the selected values. One can consider some leeway, given in the form of a second matroid. Here, the elements selected by the decision maker should be an independent set of the second matroid at all times, but the performance is measured by the set , such that is the independent set in the first matroid of maximal value that is contained in . This problem and similar ones remain as interesting future work.
1.1. Our Model
We consider a setting where can choose up to elements immediately and irrevocably, but the valuation of ’s output is then the sum of the top elements out of the chosen elements. Let , , denote the set of elements chosen by , and let be the set of the top values in . Then, the value of is .
Note that this scenario is entirely equivalent to a setting in which the decision maker can hold up to elements at any point in time, but can replace an already chosen element with a new one up to times. Indeed, an algorithm for the latter scenario can be simulated by the former scenario by selecting all elements that were selected along the process, including the ones replaced. Similarly, an algorithm for the former scenario can be simulated by the latter one by selecting the same elements, and dropping the ones with the lowest value when capacity exceeds .
In the remainder of this paper, we use the former formulation of selecting elements and deriving value from the top in the selected set. We refer to this setting as the -out-of- setting.
In the -out-of- prophet model, the performance of is taken in expectation over the product distribution (and possibly the randomness of the algorithm). The performance of the prophet is given by .
In the -out-of- secretary model, the performance of is taken in expectation over the random order of arrival (and possibly the randomness of the algorithm), when considering worst case values . This performance is measured with respect to the following benchmark: .
Given a set of non-negative numbers, let denote the subset of size of maximum total value, i.e., . We extend this definition to a distribution
over sets, and define the random variable, where is drawn according to . The total value in the set is denoted by , i.e., .111Note that although there might be several maximizers, so might not be unique, is well-defined. For simplicity, throughout this paper we assume that is a maximizer (and not the set of maximizers) and notice that our statements do not depend on the chosen maximizer. Similarly, given a distribution over sets, let .
An algorithm for the -out-of- prophet setting selects up to elements in an online fashion. Let denote the distribution over sets (of cardinality up to ) returned by , where the input is distributed according to . We say that an algorithm induces an -out-of- prophet inequality with a competitive ratio if for every product distribution it holds that
As in (Azar et al., 2014), we also consider settings in which has only limited information about the product distribution . A single-sample algorithm is one that has no information about the distribution , except for a single sample from it. A single-sample algorithm
receives as input two vectors: (a) a vector ofsamples, distributed according to , which is received offline, and (b) a vector of values, distributed according to , which is received in an online fashion. A single-sample algorithm for the -out-of- prophet setting selects up to elements (drawn from ) in an online fashion, knowing the sample vector from the outset. We denote the distribution over sets (of cardinality up to ) returned by by . A single-sample algorithm is said to induce an -out-of- prophet inequality with a competitive ratio if
A threshold algorithm for the -out-of- prophet setting, after observing some of the input elements but before accepting any of them, decides on a threshold ; After this decision was made, accepts the first elements for which . We see this single decision property as a desired simplicity property, and note that all the -out-of- prophet algorithms we present here are threshold algorithms. A specific case of the above two is single-sample threshold algorithms, in which the threshold is a function of the (offline) sample vector.
Note that all our algorithms are order oblivious. Therefore, the same guarantees hold when an adversary chooses separately the order of samples and the order of values.
An algorithm for the -out-of- secretary setting selects up to elements in an online fashion, where the values are chosen by an adversary, but the values arrive at a uniformly random order. Let denote the distribution over sets (of cardinality up to ) returned by . We say that an algorithm induces an -out-of- secretary inequality with a competitive ratio if for every vector of values it holds that
1.2. Our Results and Techniques
In this section we state our lower and upper bounds on the competitive ratios for the -out-of- prophet and secretary problems. These results are summarized in Tables 2 (prophet settings) and 2 (secretary settings), where the numbers in parentheses refer to the corresponding theorem numbers.
|Lower bound||Upper bound|
|Lower bound||Upper bound|
Our first theorem (Theorem 2.1) provides a single-sample algorithm for the prophet setting.
Theorem [single-sample, prophet, positive result]: There exists a single-sample algorithm that induces an -out-of- prophet with competitive ratio .
The algorithm is a single-threshold algorithm, which sets a threshold that equals the highest sample, and accepts all values exceeding this threshold, up to reaching capacity . We prove that this threshold algorithm exhibits two properties. First, it chooses all the highest values with high probability. Second, we show that for every product distribution, if the algorithm does not choose the highest values, it does not lead to a large loss. On the other hand, we show (in Theorem 2.4) that this result is tight if the number of possible returns is linear in (i.e., ).222When the number of possible returns is small (, for some constant ), our result is not tight; in fact, the original algorithm of (Alaei, 2014, Thm. 4.8) (without returns) gives a better bound.
Theorem [single-sample, prophet, negative result]: No single-sample algorithm obtains a competitive ratio higher than .
To establish the negative result, we first observe that a single-sample algorithm must select every value that significantly exceeds all samples and all previously observed values, so as to obtain a good competitive ratio. Therefore, no single-sample algorithm can handle a scenario in which there are more than values with this property. To conclude the result, we establish a product distribution that exhibits this scenario with sufficiently high probability.
We next present a deterministic algorithm for the -out-of- prophet setting that improves a result obtained by Assaf and Samuel-Cahn (2000). Assaf and Samuel-Cahn study a setting where instead of running a single online algorithm on the sequence, one is allowed to run simultaneous online algorithms, each selecting a single element; the performance of the algorithm is then measured by the maximum value obtained by any of the algorithms running in parallel. They establish the existence of an algorithm that obtains a competitive ratio of . The information held by the algorithm is the distribution of the maximum value, .
We improve their result by presenting an algorithm which obtains a competitive ratio of , improving the competitive ratio from decreasing proportionally to to decreasing exponentially in . Our algorithm has the same information assumed in (Assaf and Samuel-Cahn, 2000), namely the distribution . We obtain our result by providing a deterministic single-threshold algorithm for the -out-of- prophet setting. We then observe that any such algorithm can be simulated by simultaneous online algorithms, each selecting a single value.
The -out-of- prophet problem was also studied in (Assaf et al., 2002; Assaf and Samuel-Cahn, 2005). In these works, the authors establish lower bounds on the competitive ratio that can be achieved for any (Assaf et al., 2002) and for the case (Assaf and Samuel-Cahn, 2005). Their bounds are given as a recursive formula (and not in closed form), and are mostly approximated using numeric methods. In contrast, we give a closed form lower bound that decreases exponentially with .
Theorem [, prophet, positive result]: There exists a deterministic single-threshold algorithm that induces a -out-of- prophet with a competitive ratio . This algorithm knows only .
The algorithm sets a single threshold , such that , and selects the first elements that exceed . To establish the result, we show that the probability that more than values exceed is negligible, and that in the event that more than values exceed , we do not lose much.
Finally, Theorem 2.6 establishes a negative result for every prophet algorithm, even one that has full information on from the outset.
Theorem [prophet, negative result]: No -out-of- prophet algorithm achieves a better competitive ratio than .
The negative result is obtained by setting each distribution as having value with probability and value otherwise. We carefully set the values of to ensure that the best online algorithm always accepts each non-zero value. The asserted bound is then obtained by quantifying the loss of such an algorithm if there are more than non-zero values.
Theorem 3.2 provides an algorithm for -out-of- secretary settings.
Theorem [secretary, positive result]: There exists an algorithm that induces an -out-of- secretary with a competitive ratio .
The algorithm divides the values into segments, numbered from to . In the -th segment the algorithm accepts if it belongs to the highest values seen so far, and the capacity is not exhausted. We bound the probability that an element which belongs to the top elements is accepted by our algorithm. Our algorithm has two potential sources of loss, namely (a) elements that belong to OPT, but sufficiently many higher elements appeared before them, and (b) elements that belong to OPT that appear after the algorithm has exhausted its capacity. We bound the two losses to obtain our result.
On the negative side, we establish the following result:
Theorem [secretary, negative result]: No -out-of- secretary algorithm achieves a better competitive ratio than .
To establish this result we construct a sequence where the highest value is significantly larger than the second highest one, and show that no algorithm can choose the highest value with a probability higher than .
Implications to mechanism design
In Section 4, we derive from our -out-of- prophet algorithms implications to mechanism design for economic scenarios with overbooking. We show that given a single-threshold -out-of- prophet algorithm with competitive ratio , we can construct a truthful mechanism that obtains the same competitive ratio with respect to optimal welfare. The mechanism is composed of two phases. In the first phase, tickets for participating in the second phase are offered to buyers whose value exceeds a uniform threshold . The second phase is a VCG mechanism with reserve price . Similarly, if all values are identically and independently distributed according to a regular distribution, we construct a two-phase mechanism that obtains the same competitive ratio with respect to revenue.
1.3. Related Work
Following the seminal works on prophet (Krengel and Sucheston, 1977, 1978; Samuel-Cahn, 1984) and secretary (Gardner, 1966; Gilbert and Mosteller, 1966) problems, many new variants of these problems were studied. We hereby mention a few variants beyond the ones already discussed in Section 1. Krieger and Samuel-Cahn (2012) analyzed secretary scenarios with noisy values. Azar et al. (2014) studied scenarios in which the decision maker does not know the distributions from which values are drawn, rather she receives a single sample from each distribution. Vardi (2015) analyzed secretary scenarios in which every element repeats several times. Secretary settings with non-uniform random arrival orders were investigated by Kesselheim et al. (2015). Kesselheim et al. (2016) considered secretary settings in which commitments are temporary and the number of parallel commitments is bounded. More recently, prophet secretary scenarios were introduced and studied (Esfandiari et al., 2017; Ehsani et al., 2018; Azar et al., 2017). These are scenarios in which the decision maker knows the distributions from which values are drawn (as in prophet scenarios) and the elements arrive in a random order (as in secretary scenarios).
Common to all of these works (as well as to the ones surveyed in Section 1) is that the objective function which the decision maker wishes to maximize is the sum of the selected elements. A different branch of generalizations considers other objective functions. Gusein-Zade (1966), Gilbert and Mosteller (1966), and Frank and Samuels (1980) analyze secretary problems in which the decision maker wishes to maximize the probability of choosing one of the maximal elements; and Chow et al. (1964) and Krieger and Samuel-Cahn (2009) consider the expected rank of the selected element. Recently, motivated by questions in mechanism design, submodular and additional combinatorial valuations were also studied, for both secretary scenarios (Feldman et al., 2011; Barman et al., 2012; Bateni et al., 2013; Feldman and Zenklusen, 2015; Feldman and Izsak, 2017) and prophet scenarios (Rubinstein and Singla, 2017).
The literature on secretary and prophet inequalities has interesting implications to the design of online mechanisms and posted price mechanisms in particular. These include settings with unit-demand valuations (Chawla et al., 2010; Alaei, 2014), subadditive valuations (Rubinstein and Singla, 2017), and more general combinatorial valuations (Alaei, 2014; Feldman et al., 2015; Duetting et al., 2017), For further details, see the recent survey by Lucier (2017). Finally, Abolhassani et al. (2017) study the classic prophet inequality setting in large markets, assuming random or best arrival order.
The study of online settings with some degree of revocability has been also considered by Babaioff et al. (2009), Ashwinkumar and Kleinberg (2009), and Ashwinkumar (2011). These papers consider the buyback problem, where admitted elements can be revoked, but cancellation incurs some cost, and the goal is to maximize the net benefit. In our work, we do not model the cost explicitly; rather, we consider scenarios in which a limited number of cancellations is permitted.
2. -out-of- Prophet Inequalities
2.1. Single-Sample Algorithm
Consider the following family of single-sample threshold algorithms, parameterized by . The algorithm receives offline samples , and awards online. picks at most awards online.
Algorithm 0 ().
Offline: Choose a random permutation uniformly from . Let be the -maximal element of (sorted by lexicographic order; i.e., if or and .)
Online: Accept the first elements such that .
Remark: Note that the permutation is used for tie breaking. If is atomless for all , then there is no need for tie breaking and the algorithm simply chooses the -maximal sample in the first phase, and accepts the first awards that exceed in the second phase.
Theorem 2.1 ().
For every product distribution and : The expected sum of the top elements returned by is at least of the expected sum of the maximal elements of . I.e., achieves a competitive ratio of .
For proving this theorem, we first define an auxiliary (non-product) distribution, for , and show it is sufficient to prove the theorem for .
Let be a sequence of pairs of non-negative elements, and we define for by . We define a distribution over to be the uniform distribution over . In addition, we define to be the projection of on the last coordinates (the awards).
Lemma 2.2 ().
Let be a single-sample algorithm. If for every it holds that
then for every product distribution ,
Hence, in order to prove Theorem 2.1 it suffices to show that for every , performs well when its input is drawn according to .
First we note that if for every it holds that
Since the distribution of the input of (i.e., where ) is equivalent to , the left hand side of the inequality equals to . Similarly, where is equivalent to , so the right hand side equals . Hence, we get that . ∎
It now remains to prove that for every , achieves a good approximation with respect to .
Lemma 2.3 ().
For every , it holds that
Given , the random tie-breaking rule is equivalent to an infinitesimal perturbation of the entries of of the form for being an i.i.d. noise and being infinitesimal compared to the non-zero elements of . Note that this perturbation is equivalent to perturbing the entries of by . Since such perturbation does not change (in an essential way) the value of subsets of , we can assume w.l.o.g. that all the entries of are different from each other (and hence the elements of are strictly ordered, so the tie-breaking rule plays no role and can be ignored).
Given a vector , we define to be the highest entry in , and for simplicity we define . We use the notation for the last coordinates of (i.e., the awards). Recall that was defined as the uniform distribution over . Note that given , is deterministic and hence,
Consider the summation on the right. It holds that
The last inequality follows since whenever and , it holds that , and we sum over at most elements.
Consider the left term in Eq. (2); It holds that
where the equality holds since is the largest value, and so it is picked by the prophet if it is a reward, which occurs with probability . Substituting the last inequality in Eq. (2) gives
Substituting in Eq. (1), we get:
Given an index , we define to be the threshold set by and to be the set of awards above the threshold, . Note that if , then . Therefore,
Let be the set of the highest elements of , and let be the set
Notice that if for less than of the pairs in the greater element of the pair was chosen to be a sample, then there are less than samples in , thus . Since for each , the greater element of the pair is independently chosen to be a sample with probability , by applying Chernoff bound, we get
Similarly, by setting to be the set of the highest elements of and to be the set of indices s.t. exactly one of is in , and following the same analysis, we get that
We next show that in the case where , the bound given in Theorem 2.1 is tight with respect to any single-sample algorithm.
Theorem 2.4 ().
No single-sample algorithm can achieve a competitive ratio better than .
Let be a single-sample -out-of- algorithm and let and be two sequences s.t.
Consider the event where the sample vector is and the award vector is . Since accepts at most elements, there must be some , , s.t. rejects the element with probability at least . Let denote this index.
Suppose both samples and awards are distributed according to the following product distribution :
For , gives either or , each with probability .
For , gives with probability 1.
For , gives with probability 1.
Now, consider the event that the sample vector is as before, the award vector is
(note that both and are in the support of ) and rejects the element. Since for every , this event happens with probability at least . In this event, the difference between the optimal performance and the performance of is at least (use the convention that ), and hence
It also holds that . We get that the competitive ratio of is at most
For a sufficiently large ratio of the bound is arbitrarily close to . ∎
2.2. Algorithm Based on the Distribution of Max Value
In this section, we show a simple single-threshold algorithm for -out-of- prophet scenarios which is based only on the distribution of the maximal element, and achieves fraction of the expected value of the maximal element. Let be the distribution of the maximal element, and we assume that has no mass points for all .
Algorithm 0 ().
Set a threshold such that
Accept the first elements for which .
Theorem 2.5 ().
For every product distribution and : The expected value of the maximal element returned by is at least of the expected maximal element. I.e., achieves a competitive ratio of .
We use the notation for the expected maximal element. First, we show that for every it holds that
where the inequality follows from the inequality of arithmetic and geometric means. We can now apply Chernoff bound to get:
Next, we notice that for every it holds that and since are independent we get that for any prefix of size ,333If are independent events, and are independent events, and for all , then .
We are now ready to establish the bound on the performance of .
We observe that a single-threshold algorithm, as we analyzed here, can be translated into an algorithm in the setting of (Assaf and Samuel-Cahn, 2000) as follows. Our algorithm sets a threshold and pick the first elements that exceed . To apply our algorithm to their setting, let (for ) be the algorithm that accepts the element that exceeds . One can easily verify that a competitive ratio for our setting carries over to their setting. Our result (Thm. 2.5) shows that with exactly the same limited information as (Assaf and Samuel-Cahn, 2000), , one can get an improved competitive ratio, , that decreases exponentially with .444Note that our algorithm translates to semi-threshold algorithms, where the element that exceeds the threshold is selected, rather than the first one.
We also note that algorithm can be modified slightly to handle cases in which might contain mass points.555The modified algorithm is not a single-threshold algorithm, but it can still be simulated by simultaneous algorithms. In order to do so, we make the following modifications: (a) instead of setting such that , we set . (b) the algorithm accepts the first element such that . (c) thereafter, the algorithm accepts the first elements such that . A similar analysis shows that the modified algorithm guarantees a competitive ratio of