Constrained submodular maximization has attracted growth attention recently calinescu2011maximizing buchbinder2018deterministic buchbinder2014submodular . Most existing work on submodular maximization focus on selecting a subset of items subject to given constraints so as to maximize a submodular objective function krause2008near . In this paper, we study adaptive robust optimization with nearly submodular structure (ARONSS). This study belongs to the category of robust submodular maximization. Our objective is to randomly select a subset of items that performs well over several reward functions. Although robust submodular maximization has been well studied anari2017structured krause2008robust chen2016robust orlin2018robust
, most of existing studies assume an non-adaptive setting, i.e., one has to select a subset of items all at once in advance, and submodular reward function. However, in many applications from artificial intelligence, the outcome of an objective function is often uncertain, one needs to make a sequence of decisions adaptively based on the outcomes of the previous decisionsgolovin2011adaptive . Moreover, the reward function is not necessarily submodular. This motivates us to study the adaptive robust optimization problem with general reward functions.
The main contribution of this paper is three-fold:
We extend the previous studies on robust submodular maximization in two directions: (1) we consider the robust optimization problem under the adaptive setting, i.e., one can select one item at a time and observe the outcome of picked items, before selecting the next item, and (2) our results apply to a broad range of reward functions characterized by -nearly submodular function.
We first analyze the adaptivity gap of ARONSS and show that the gap between the best adaptive solution and the best non-adaptive solution is bounded. This enables us to focus on designing non-adaptive solutions which are much easier to work with.
Then we propose an approximate solution to this problem when all reward functions are submodular. The approximation ratio is when considering matroid constraint. We also present two algorithms that achieve bounded approximation ratios for the general case. All algorithms are non-adaptive and easy to implement.
2 Preliminaries and Problem Formulation
2.1 Submodular Function
A set function that maps subsets of a finite ground set to non-negative real numbers is said to be submodular if for every with and every , we have that
A submodular function is said to be monotone if whenever .
2.2 Items and States
Let denote a finite set of items, and each item is in a particular state from a set of possible states. Let denote a realization of item states. Each item
is associated with a random variablethat represents a random realization of ’s state. We use
to denote the collection of all variables. We assume there is a known prior probability distributionover realizations for each item , i.e., . We further assume that the states of all items are decided independently from each other, i.e., is drawn randomly from the product distribution . We use to denote the realization of items’ states. After picking an item , we are able to observe its state .
2.3 -nearly Submodular Reward Functions
We are given a family of reward functions , where each maps a set of items and their states to some reward . In this work, we assume each function is monotone, i.e., for all , and -nearly submodular, i.e., for any , there is a submodular function such that for any , we have where . It is easy to verify that any submodular function is -nearly submodular.
2.4 Adaptive Policies
We model the adaptive strategy of picking items through a policy golovin2011adaptive . Formally, a policy is a function that specifies which item to pick next under the observations made so far: . Note that
can be regarded as some decision tree that specifies a rule for picking items adaptively. Assume that when the items are in state, the policy picks a set of items (and corresponding states), which is denoted by . Thus, given the policy , its expected reward received from function is . In the context of robust optimization, our goal is to pick a set of items (and corresponding states) that achieves high reward in the worst-case over reward functions in . Thus, we define the utility of as
Let be a downward-closed family of subsets of , i.e., a family of subsets is downward-closed if for any subset in , it also belongs to . We use to refer to the subset of items picked by policy given state . We say a policy is feasible if for any , . This downward-closed family generalizes many useful constraints such as matroid and knapsack constraints. Our goal is to identify the best feasible policy that maximizes its expected utility.
3 Analysis on Adaptivity Gap
We say a policy is non-adaptive if it always picks the next item independent of the states of the picked items. Clearly adaptive polices obtain at least as much utility as non-adaptive policies. Perhaps surprisingly, building on recent advances in stochastic submodular probing bradac2019near , we show that this adaptivity gap is upper bounded by a constant (given that is a constant). Based on this result, we can focus on designing non-adaptive polices which are much easier to work with.
Given any adaptive policy , there exists a non-adaptive algorithm such that .
Proof: Given any adaptive policy , we follow the idea in gupta2017adaptivity and define a non-adaptive policy
: randomly draw a state vectorfrom the product distribution (this step is done virtually), pick , i.e., pick all items picked by given . Let be the state of all items drawn virtually by and be the true state of all items when picked by .
Now consider any , the expected value of obtained by is
Because is -nearly submodular, we have
We next analyze the utility of . The expected value of obtained by is
Because is -nearly submodular, we have
It follows that
It was worth noting that Theorem 1 holds when is a prefix-closed family of constraints, i.e., a family of subsets is prefix-closed if for any subsequence in , its prefix also belongs to .
4 Approximate Solution for Submodular Reward Function
We first focus on the case when , i.e., all reward functions are submodular. We propose a constant approximate solution to this special case. The basic idea of our approach is that we first derive a constant approximate solution to the non-adaptive robust optimization problem and Theorem 1 implies that this solution is also a constant approximate solution to the original problem.
We first introduce the non-adaptive robust optimization problem with submodular structure. Given any reward function , we use to denote the expected reward of selecting . Given a non-adaptive policy , let denote the expected reward gained from function where is the probability that is selected by . The utility of is . We next formulate the non-adaptive robust optimization problem as follows.
Before introducing our algorithm, we first introduce some important notations. For a independence system , the polytope of is defined as where denotes the vector with entries one and all other entries zero. Given a vector , the multilinear extension of is defined as . Define the marginal of for as where denotes the component wise maximum.
As a corollary of Theorem 1, i.e., when , the following lemma bounds the adaptivity gap when all reward functions are submodular.
Let denote the optimal adaptive policy and denote the optimal non-adaptive policy, we have .
We next propose a continuous greedy algorithm that achieves a constant approximation ratio of P.1. We follow the framework of chekuri2010dependent to derive the following lemma.
Given submodular functions and a value , independence system , the continuous greedy algorithm finds a point such that or outputs a certificate that there is solution with .
Proof: Consider any vector . If there exists policy, say , such that , we have
In other words, for any fractional solution , there exits a direction where the entry of is such that
. And this direction can be found using linear program. We follow the continuous greedy algorithm and obtain a solutionsuch that .
If such policy does not exist, we output a certificate that there is feasible solution that achieves utility .
Based on Lemma 2, we can perform a binary search on to find a -approximate fractional solution. At last, depending on the type of , we use an appropriate technique to round the fractional solution to an integral solution. Lemma 1 and Lemma 2 imply the following main result.
Our algorithm returns a solution that achieves approximation ratio where is the performance loss due to rounding.
5 Two Heuristics for Nearly Submodular Reward Functions
In this section, we introduce two algorithms for computing approximate solutions for the general case. Since the adaptivity gap is bounded in Section 3, we focus on building non-adaptive policies. In the rest of this paper, we use to denote a non-adaptive policy.
5.1 A -approximate Solution
The basic idea of the first algorithm (Algorithm 1) is very simple, we first solve for each , then randomly pick one among outputs as solution. Since we focus on designing non-adaptive solutions, for notation convenience, define as the expected value of obtained from picking (irrespective of items’ states), i.e., . One can verify that solving is equivalent to solving .
To carry out these steps, requires one oracle which returns an approximate solution to for each . Assume the approximation ratio of is , we have
Assume is the optimal adaptive policy and , our first policy achieves approximation ratio for ARONSS, i.e., . The time complexity of is where is the time complexity of .
Proof: First, according to the definition of , for any , we have
Based on the design of , is returned as the final solution with probability . Because achieves approximation ratio , we have , it follows that . Thus,
Discussion on the value of
We next briefly discuss possible solutions to . Consider a special case when all reward functions in are submodular, i.e., , and is a family of subsets that satisfies a knapsack constraint or a matroid constraint calinescu2011maximizing , there exist algorithms that achieve approximation ratio, i.e., . For more complicated constraints such as intersection of a fixed number of knapsack and matroid constraints, chekuri2014submodular provide approximate solutions via the multilinear relaxation and contention resolution schemes.
5.2 Double-Oracle Algorithm
We next present a double-oracle based solution to ARONSS. We first introduce an optimization problem P.2 as follows.
In P.2, indicates the probability of picking . It is easy to verify that finding is equivalent to solving P.2. In practice, P.2 is often solved by the double oracle algorithm mcmahan2003planning . Without loss of generality, assume that double oracle algorithm finds a approximate solution to P.2, i.e., , we have due to the adaptivity gap proved in Theorem 1.
Assume finds a approximate solution to P.2, achieves approximation ratio for ARONSS, i.e., .
As compared with , we remove from the above approximation ratio, however, the time complexity of could be exponential.
To the best of our knowledge, we are the first to systematically study the problem of adaptive robust optimization with nearly submodular structure. We analyze the adaptivity gap of ARONSS. Then we propose a approximate solution to this problem when all reward functions are submodular. Our algorithm achieves approximation ratio when considering matroid constraint. At last, we develop two algorithms that achieve bounded approximation ratios for the general case.
- (1) G. Calinescu, C. Chekuri, M. Pál, J. Vondrák, Maximizing a monotone submodular function subject to a matroid constraint, SIAM Journal on Computing 40 (6) (2011) 1740–1766.
- (2) N. Buchbinder, M. Feldman, Deterministic algorithms for submodular maximization problems, ACM Transactions on Algorithms (TALG) 14 (3) (2018) 32.
- (3) N. Buchbinder, M. Feldman, J. S. Naor, R. Schwartz, Submodular maximization with cardinality constraints, in: Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms, Society for Industrial and Applied Mathematics, 2014, pp. 1433–1452.
A. Krause, A. Singh, C. Guestrin, Near-optimal sensor placements in gaussian processes: Theory, efficient algorithms and empirical studies, Journal of Machine Learning Research 9 (Feb) (2008) 235–284.
- (5) N. Anari, N. Haghtalab, S. Pokutta, M. Singh, A. Torrico, et al., Structured robust submodular maximization: Offline and online algorithms, arXiv preprint arXiv:1710.04740.
- (6) A. Krause, H. B. McMahan, C. Guestrin, A. Gupta, Robust submodular observation selection, Journal of Machine Learning Research 9 (Dec) (2008) 2761–2801.
- (7) W. Chen, T. Lin, Z. Tan, M. Zhao, X. Zhou, Robust influence maximization, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2016, pp. 795–804.
- (8) J. B. Orlin, A. S. Schulz, R. Udwani, Robust monotone submodular function maximization, Mathematical Programming 172 (1-2) (2018) 505–537.
D. Golovin, A. Krause, Adaptive submodularity: Theory and applications in active learning and stochastic optimization, Journal of Artificial Intelligence Research 42 (2011) 427–486.
- (10) D. Bradac, S. Singla, G. Zuzic, (near) optimal adaptivity gaps for stochastic multi-value probing, arXiv preprint arXiv:1902.01461.
- (11) A. Gupta, V. Nagarajan, S. Singla, Adaptivity gaps for stochastic probing: Submodular and xos functions, in: Proceedings of the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, 2017, pp. 1688–1702.
- (12) C. Chekuri, J. Vondrak, R. Zenklusen, Dependent randomized rounding via exchange properties of combinatorial structures, in: 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, IEEE, 2010, pp. 575–584.
- (13) C. Chekuri, J. Vondrák, R. Zenklusen, Submodular function maximization via the multilinear relaxation and contention resolution schemes, SIAM Journal on Computing 43 (6) (2014) 1831–1879.
- (14) H. B. McMahan, G. J. Gordon, A. Blum, Planning in the presence of cost functions controlled by an adversary, in: ICML, 2003, pp. 536–543.