1 Introduction
In this paper we consider combinatorial problems of the form
with
and uncertain cost vector
. To find a solution that still performs well under the possible cost realizations, different approaches have been proposed. These include fuzzy optimization [KZ10], stochastic programming [BL11], or robust optimization [GS16, GMT14].In the robust optimization approach, we assume that all possible cost realizations are modelled through a socalled uncertainty set
, and we want to protect against all outcomes without knowledge of a probability distribution. The resulting worstcase problem is then of the form
Special cases of this kind have been investigated thoroughly (see [KZ16] for a recent overview on robust combinatorial optimization). A common assumption in these settings is that the shape of the uncertainty set is known, e.g., one assumes that the uncertainty is intervalbased
discrete
or ellipsoidal
Note that in the minmax setting, using a discrete uncertainty set is equivalent to using its convex hull . So far, comparatively little research has investigated how a decision maker can actually come up with an uncertainty set that produces a robust solution in accordance with his or her wishes. In the datadriven approach [BGK18], we do not assume an uncertainty set to be given, but only data observations, which are usually discrete. These are then used to construct uncertainty sets, e.g, using different approaches from statistical testing.
In [CG18c, CG18a], the authors considered a setting in which the shape of the uncertainty set is given, but not its size. Models are introduced by which compromise robust solutions can be found, which perform well on average over all considered uncertainty sizes. Furthermore, in [CKR18] the sensitivity of robust solutions to the uncertainty size was considered.
In the recent paper [CDG19], realworld data modeling traffic speeds in the city of Chicago were used to test different uncertainty sets for shortest path problems experimentally. In particular, discrete uncertainty and ellipsoidal uncertainty sets were found to produce a good tradeoff on an outofsample evaluation set of scenarios with respect to average performance, worstcase performance and the conditional value at risk (CVaR) criterion.
In this paper, we introduce a novel approach to handle uncertainty in robust optimization. This mixeduncertainty setting is a direct generalization of the classic robust optimization approach, where we protect against multiple uncertainty sets simultaneously. We demonstrate that this approach is wellsuited to datadriven settings, where the decision maker is not able to determine the shape and size of uncertainty a priory. By using irace [LIDLSB11], a software designed to tune algorithms automatically, we determine the bestperforming combination of uncertainty sets for the same Chicago test data as used in [CDG19]. Our proposed approach is generally applicable, and possible to implement using offtheshelf software with little theoretical knowledge from the decision maker, making it flexible and attractive for practical purposes.
The rest of this paper is structured as follows. In Section 2, we formally introduce our setting of mixed uncertainty sets for uncertain combinatorial problems. We demonstrate how to find compact mixedinteger programming models, and point out cases solvable in polynomial time. Section 3 presents an extensive computational case study, highlighting the usefulness of our approach for a realworld shortest path problem. We conclude this paper in Section 4 and point out further research questions.
2 Mixed Uncertainty Sets
In classic minmax robust combinatorial optimization, we consider the problem
where the uncertainty set contains all possible outcomes of the cost vector . As a direct generalization, it is possible that multiple uncertainty sets need to be considered. For each , we are given a weight that denotes the importance to protect against , or likelihood of its occurrence.
The resulting weighted robust optimization is then as follows:
There are different settings where the application of a model of the form (WRP) can be useful. For example: (1) We receive different forecasts for future developments of costs, each providing a set of most likely scenarios. Instead of using only one of these sets or merging them, we consider a weighted robust problem, where we assign each forecast an expert estimate
whether it can be trusted. (2) We create different uncertainty sets based on different degrees of riskwillingness. We then find a single compromise solution against all levels of risk. For example, one uncertainty set may cover the worst 80% of outcomes of a multivariate normal distribution, whereas another uncertainty set covers 95%. We can find a single solution hedging against both worst cases with prescribed weights. (3) We have an original set of observations for our uncertain data, and do not know which shape of uncertainty set may be appropriate for the problem at hand. Selecting the (right) uncertainty set is itself a decision problem under uncertainty. So far, researchers left this problem to the user/decisionmaker and developed approaches for robust optimization after this decision is made offering different choices to this decision. However, this itself can be posed as an optimization or a learning problem. In other words, ideally, we would want to learn the right uncertainty set from data, which may be a mix of many known sets. Also, automatically learning a mixed uncertainty set allows the possibility for the set to be dynamic which can change with a change in the random nature of the underlying uncertain parameters.
Mixing uncertainty sets and tuning the mixing (hyper) parameters to learn the best mix is a challenging task. Research in this direction is in its early stages, the only work to have experimented in this direction is that of [CH15]
where authors consider combining ellipsoidal sets within a Bayesian setting for robust linear programs. Moreover, even under the assumption that we can formulate this learning task and embed it in a holistic algorithmic framework for datadriven robust optimization, there is no guarantee that mixing sets will help in improved robust solutions for any choice of robustness measures. In this spirit, (WRP) is a first step towards automating selection of the right uncertainty set. In this work we would like to investigate if mixing (known) sets can give better robust solutions.
We now consider the complexity of (WRP), and possibilities to formulate it using mixedinteger programming. Note that (WRP) can be written as a compact optimization problem if that is possible for each , by combing each model. As an example, if is a budgeted uncertainty set of the form (see [BS03]), then the robust problem (RP) can be written as
s.t.  
If is a polyhedral uncertainty set of the form , then its robust problem (RP) can be written as
s.t.  
Combining both uncertainty sets in a weighted robust problem (WRP) using weights for budgeted uncertainty and for polyhedral uncertainty, the combination of models yields
s.t.  
As (WRP) is an generalization of (RP), it is at least as hard. Hence, cases of (WRP) that involve uncertainty sets for which we already know that the classic robust optimization problem is NPhard, will be NPhard as well.
We therefore focus on cases where (RP) is still polynomially solvable. First, notice that when are interval uncertainty sets of the form , then
which is a problem of nominal type. Hence, (WRP) with interval sets has the same complexity as the nominal problem (P).
Now let be budgeted uncertainty sets with . We can write (WRP) as
(BWRP)  
s.t.  
Lemma 1.
There is an optimal solution to (BWRP), where for all , we have for some , or .
Proof.
For some fixed , the remaining optimization problem in and can be decomposed in to independent subproblems. For each subproblem, the result is known from the classic budgeted uncertainty case (see [BS03]). ∎
Theorem 2.
For constant , (BWRP) can be solved in polynomial time, if (P) can be solved in polynomial time.
Proof.
We rewrite (BWRP) as
s.t. 
where . Note that
Using Lemma 1, we enumerate all options for the variables. For fixed values of , (BWRP) reduces to
which is of the nominal type. ∎
Note that this approach cannot be used if is unbounded (i.e., part of the input). For this case, we can still identify cases that are tractable in the following.
For , denote by the corresponding set of chosen items. We rewrite the objective of (WRP) as
where denotes the sum of the largest values for , i.e.,
Theorem 3.
Function is submodular.
Proof.
We first show that is a submodular function. As the sum of submodular functions is still submodular, it follows that is submodular.
To see that is submodular, let be given. Let be the up to items that maximize . Note that , while is not necessarily in . Figure 3 illustrates these relationships.
For any element , one of two cases hold: If , then also and . If , then there is an element with . Hence,
and the claim follows. ∎
Corollary 4.
For problems where is a matroid, (BWRP) can be solved in polynomial time, even for unbounded .
Corollary 4 applies to, e.g., the weighted robust spanning tree problem, or the weighted robust selection problem.
Note that these results also apply to combinations of interval and budgeted uncertainty sets.
Finally, we consider the approximability of problems with discrete uncertainty sets. It is a wellknown result (see, e.g., [ABV09, CG18b]) that for classic robust combinatorial optimization problems with discrete uncertainty , an optimal solution to the midpoint scenario gives a approximation. We extend this result to mixed discrete uncertainty sets.
Let for all . We construct the mixeduncertainty midpoint scenario
and set .
Theorem 5.
An optimal solution to problem (P) with costs is a approximation to (WRP) with mixed discrete scenarios.
Proof.
Let be an optimal solution to , and let be an optimal solution to (WRP). Then
Hence, the objective of solution in problem (WRP) is at most times the objective of an optimal solution. ∎
3 Tuning Experiments
Algorithms which solve optimization problems involve a number of parameters, and these parameters can be carefully tuned so that the performance of the algorithms is optimized. For this purpose, a tuning tool to automatically configure optimization algorithms called irace [LIDLSB11] has been developed. We use irace to find the bestperforming combination of uncertainty sets for (WRP) a given set of instances of the robust shortest path problem.
3.1 Experimental Setup
We used the same realworld shortest path data as first introduced in [CDG19]. The graph models the City of Chicago, and consists of 538 nodes and 1308 arcs. The morning data set was used to model the problem uncertainty, containing 271 scenarios that represent morning rush hours during week days. Each scenario contains the travel speed for each arc.
In our experiments, three uncertainty sets, ellipsoidal, convex hull and interval, are used in generating the mixed uncertainty sets. Additionally, we set , meaning that we combine up to three uncertainty sets; however, the tuning algorithm can also choose less. We refer to these three sets as the parent uncertainty sets. The choice of parent uncertainty sets is driven by computational efficiency and relative performance of uncertainty sets demonstrated in [CDG19]. It is possible that two parent sets are of the same type, e.g., a combination that uses two convex hull uncertainty sets and one ellipsoidal uncertainty set. Corresponding to each parent set, we have a scaling parameter , which takes up values in predefined intervals (including the interval limits). The intervals are so chosen to reflect a reasonable range of choices for a decision maker. For ellipsoidal uncertainty we use , while we use for interval and convex hull uncertainty. Parameter represents the size of the uncertainty set, in relation to the observed scenarios. For a formal parameter definition, we refer to [CDG19].
Moreover, we also associate a weight to each parent set which corresponds to the significance of the parent in the mixed uncertainty set and, therefore, in the objective function.
In our experiments, our objective is to find a path that is robust when driving during morning rush hours, modeled through 271 scenarios. We use 75% of the scenarios (insample data) to construct the uncertainty sets, and we evaluate solutions insample and outsample separately. To this end, we generate 600 random st pairs over the node set, which fulfill a minimum distance criterion to avoid nodes that are close to each other. To find a balanced evaluation of all methods, we use three performance criteria: (1) The average objective function value over all st pairs and all scenarios, denoted as Avg. (2) The average of the worstcase objective function value for each st pair, denoted as Max. (3) The average value of the worst 5% of objective values for each st pair (as in the conditional value at risk measure), denoted as CVaR. We see that each evaluation is comprised of three performance measures, i.e., each solution is assigned a threedimensional objective vector.
For irace to tune the algorithmic parameters of mixed uncertainty sets, the algorithm must return the cost function as a single value. Hence, three additional parameters were defined, with each parameter being a weight to each performance measure in the objective vector. The cost function then becomes a weighted sum of the performance measures. However, it is important to note that these three weights are userdefined and not automatically configured by irace. The three weights corresponding to their performance measures are sampled from the interval with a step size of , and only those values are retained that sum up to one (which makes a total of 66 such combinations of weights). Each combination results in an algorithm, and therefore, a solution for the shortest path problem using the mixed uncertainty sets. Hence, in total, for mixed uncertainty sets, we have nine automatically configurable parameters and three fixed userdefined parameters. We used a fixed computational budget of 10,000 experiments for a given irace run.
As a comparison, we generate 41 possible values for the scaling parameter when each parent uncertainty set is separately used to compute solutions to the robust shortest path problem; this does not involve any parameter tuning as solutions are computed for different sizes of the parent uncertainty sets. The 41 values are equidistant, i.e., for ellipsoidal sets we use a step size of for , whereas for interval and convex hull we use a step size of . We use the 41 models obtained for each parent set to compare the performance of mixed set solutions with the performance of the solutions delivered by the pure parent sets.
3.2 Results
We present the performance of the uncertainty sets, both mixed and parent, in Figures 2 and 3. As mentioned earlier, a total of 41 objective space vectors are obtained for each parent uncertainty set, where as a total of 66 objective space vectors are obtained for the mixed uncertainty sets. Each objective space vector among the 66 vectors for mixed uncertainty sets corresponds to a unique best parametric configuration obtained for different combinations of the weights of the performance measures. Moreover, each element (performance measure) of the objective space vector corresponding to a unique configuration is an average of all the values of that performance measure taken over all the scenarios and st pairs. This also holds true for the solutions obtained using the parent uncertainty sets. Figure 2 shows the tradeoff curves between the Avg and the Max performance measures for both insample and outsample data, while Figure 3 shows the tradeoff curves between the Avg and the CVaR performance measures for both insample and outsample data. All the values are in unit of minutes; a lower value indicates better performance. Hence, good tradeoff solutions should move from top left to the bottom right of the curves. Some values for interval sets are not visible, as they are outside the plotted ranges.
Figure 2(a) shows that for insample performance, convex hull solutions dominate all other solutions for the Avg vs Max tradeoffs by construction; however, the mixed set solutions closely match the convex hull solutions. Interval solutions perform the worst among all, especially for higher values of the scaling parameter. Figure 3(a) shows that for insample performance, ellipsoidal solutions exhibit the best Avg vs CVaR tradeoffs among all the solutions but are closely matched by the mixed set solutions. However, convex hull solutions perform worse than both ellipsoidal and mixed set, and interval solutions closely match the performance of the convex hull solutions. Summarily, both ellipsoidal and convex hull solutions do not exhibit stability across both the tradeoff curves, i.e., while convex hull dominates the insample Avg vs Max curves, ellipsoidal exhibits best performance for insample Avg vs CVaR tradeoffs. Mixed uncertainty set solutions exhibit stability across both insample Avg vs Max and insample Avg vs CVaR tradeoffs as they closely match the respective performance of both convex hull and ellipsoidal solutions.
When outsample performance is considered for both Avg vs Max and Avg vs CVaR tradeoffs (Figures 2(b) and 3(b)), mixed set solutions can dominate all other solutions. Convex hull solutions perform even worse than interval solutions for both the tradeoffs, implying that they are overfitted to the data. The performance of ellipsoidal solutions improve from insample to outsample compared to the performance of convex hull and interval solutions for Avg vs Max tradeoffs, but lose their best tradeoff performance for Avg vs CVaR only to mixed set solutions.
The key aspect to note is that, even though mixed solutions only use the three parent sets, their combination can outperform each separately, i.e., we can observe a synergy effect when mixing uncertainty sets.
While the performance comparison among all the uncertainty sets help us establish that mixed uncertainty sets not only exhibit stability over both insample and outsample data but also perform better than the parent sets for certain configurations, we do not observe how each uncertainty set performs for each st pair separately, as we explained earlier that each solution delivered by an uncertainty set is an average of all the values taken over all the scenarios and st pairs. As an instance, to understand how each uncertainty set performs for each st pair, we choose a representative configuration for each uncertainty set to compare the performance. The description of the configurations which are compared can be found below.

Mixed Uncertainty Set: We use a mix of convex hull and ellipsoidal uncertainty. The scaling parameter, , and the weight parameter, , of convex hull are 0.2234 and 0.7502 respectively. For the ellipsoidal set, the scaling and weight parameters take the values 5.4609 and 0.9796 respectively.

Convex Hull: The scaling parameter is 0.2

Ellipsoidal: The scaling parameter is 6.0

Interval: The scaling parameter is 0.075
Figures 4, 5, 6 show the comparison in performance for each measure (Avg, Max and CVaR), both insample and outsample, for every pair of mixed set and parent set, i.e., we compare performance for mixed and convex hull sets, for mixed and ellipsoidal sets and for mixed and interval sets. This is achieved by taking the difference in the solutions delivered by the mixed set and the comparing parent set for each st pair; all the values are in unit of minutes. While a negative value indicates that the solution delivered by the mixed set is better than the parent set, a positive value indicates the opposite. For the sake of clarity in the plots, we filter out the sample containing only zero values as they indicate that both the sets deliver exactly the same solution. In our case, for each pair of mixed and parent set, the number of st pairs for which the solutions differed are as follows: 131 (21.8%) for mixed and convex hull sets; 91 (15.2%) for mixed and ellipsoidal sets; and 129 (21.5%) for mixed and interval sets.
Figure 4 compares the Avg performance measure (both insample and outsample) for each pair of mixed and parent set. We see that at least 50% of the insample solutions given by convex hull is better than that given by the mixed uncertainty set. For ellipsoidalmixed set and intervalmixed set pairs, at least 50% of the insample solutions given by mixed set is better than those of the parent sets. Moreover, for insample performance, the difference in the values of the performance measures (margin) by which convex hull solutions are better is higher than the margin by which mixed set solutions are better. For the other two pairs of mixed and parent sets, mixed set performs better by higher margins than by the margin when it performs worse. For outsample performance, mixed set solutions still dominate the solutions of ellipsoidal and interval, in both number and value. However, convex hull solutions although dominate for at least 50% of the solutions, the mixed set solutions dominate in margin of performance, i.e. when mixed set solutions are better for outsample data, they are better by a higher margin.
Similar analysis for the Max performance measure from Figure 5 leads us to conclude that while at least 50% of the insample solutions given by convex hull is not only better than those given by mixed set but also better by higher margin, for outsample data, mixed set solutions dominate convex hull solutions, in both number and margin. For ellipsoidalmixed set and intervalmixed set pairs, insample solutions given by mixed set is better than those of the parent sets, in both number and margin. For outsample performance, mixed set solutions still dominate the solutions of interval set, in both aspects, but now ellipsoidal solutions are found to be slightly dominating the mixed set solutions, in both aspects. Figure 6 compares the CVaR performance measure (both insample and outsample) for each pair of mixed and parent set. We see that mixed set solutions dominate the convex hull and interval solutions in both aspects of number and margin across both insample and outsample data. However, ellipsoidal solutions are found to be clearly dominant over the mixed set solutions, in both number and margin, across both insample and outsample data.
We summarize our findings from our experiment on the robust shortest path problem with realworld data.
Convex hull solutions show good insample performance for Avg vs Max tradeoffs, but are not stable when facing outsample scenarios. The insample performance of convex hull solutions is closely matched by the mixed set solutions. Convex hull solutions perform worse for Avg vs CVaR tradeoffs across both insample and outsample scenarios.
Interval solutions do not perform well in general, but are easy to find and can be a reasonable approach for smaller values of the scaling parameter.
Ellipsoidal solutions exhibit stability over both insample and outsample performance for both Avg vs Max and Avg vs CVaR tradeoffs and offer a good approach over a wide range of the scaling parameter. In addition, they also deliver the best CVaR performance across both insample and outsample data among all the uncertainty sets. However, solutions given by ellipsoidal set are dominated by the mixed set solutions in the region of best tradeoffs for both insample and outsample scenarios when averaged over all the st pairs and scenarios.
Mixed set solutions closely match the insample performance of convex hull and ellipsoidal solutions for Avg vs Max and Avg vs CVaR tradeoffs respectively, but they dominate all the other solutions when facing outsample scenarios, especially in the region of best tradeoffs, i.e., mixed set solutions are found to deliver the best solutions among all the uncertainty sets for certain configurations. Besides, mixed set solutions exhibit stability across both insample and outsample scenarios.
Mixed set solutions do not always deliver the best solution compared to the parent sets for each st pair when we consider the best tradeoffs among the performance measures; but when the mixed sets give better solutions than the parent sets, the margin by which they are better is much higher than the margin by which they are worse when they underperform compared to the parent sets. This makes the average value of the solutions given by mixed sets over all the st pairs and scenarios better than the parent sets, and hence, make them a better option to find better tradeoffs among the objective vector elements.
4 Conclusion
In this paper we have proposed a mixed uncertainty set approach to robust combinatorial optimization. Our results give a strong evidence in support of mixed sets giving superior solutions compared to individual approaches. This evidence paves way to further investigation into developing efficient algorithmic framework for building such mixed sets. For example, as an immediate future extension of this work, one option is to consider a Bayesian type approach, instead of a blackbox optimizer like irace, which updates the weight on each parent set under consideration with every new bit of data collected.
References
 [ABV09] Hassene Aissi, Cristina Bazgan, and Daniel Vanderpooten. Min–max and min–max regret versions of combinatorial optimization problems: A survey. European journal of operational research, 197(2):427–438, 2009.
 [BGK18] Dimitris Bertsimas, Vishal Gupta, and Nathan Kallus. Datadriven robust optimization. Mathematical Programming, 167(2):235–292, 2018.
 [BL11] John R Birge and Francois Louveaux. Introduction to stochastic programming. Springer Science & Business Media, 2011.
 [BS03] Dimitris Bertsimas and Melvyn Sim. Robust discrete optimization and network flows. Mathematical programming, 98(13):49–71, 2003.
 [CDG19] André Chassein, Trivikram Dokka, and Marc Goerigk. Algorithms and uncertainty sets for datadriven robust shortest path problems. European Journal of Operational Research, 274(2):671–686, 2019.
 [CG18a] André Chassein and Marc Goerigk. Compromise solutions for robust combinatorial optimization with variablesized uncertainty. European Journal of Operational Research, 269(2):544–555, 2018.
 [CG18b] André Chassein and Marc Goerigk. On scenario aggregation to approximate robust combinatorial optimization problems. Optimization Letters, 12(7):1523–1533, 2018.
 [CG18c] André Chassein and Marc Goerigk. Variablesized uncertainty and inverse problems in robust optimization. European Journal of Operational Research, 264(1):17–28, 2018.
 [CH15] Trevor Campbell and Jonathan P How. Bayesian nonparametric set construction for robust optimization. In American Control Conference (ACC), 2015, pages 4216–4221. IEEE, 2015.
 [CKR18] Giovanni P Crespi, Daishi Kuroiwa, and Matteo Rocca. Robust optimization: Sensitivity to uncertainty in scalar and vector cases, with applications. Operations Research Perspectives, 5:113–119, 2018.
 [GMT14] Virginie Gabrel, Cécile Murat, and Aurélie Thiele. Recent advances in robust optimization: An overview. European journal of operational research, 235(3):471–483, 2014.
 [GS16] Marc Goerigk and Anita Schöbel. Algorithm engineering in robust optimization. In Algorithm engineering, pages 245–279. Springer, 2016.
 [KZ10] Adam Kasperski and Paweł Zieliński. Minmax regret approach and optimality evaluation in combinatorial optimization problems with interval and fuzzy weights. European Journal of Operational Research, 200(3):680–687, 2010.
 [KZ16] Adam Kasperski and Paweł Zieliński. Robust discrete optimization under discrete and interval uncertainty: A survey. In Robustness analysis in decision aiding, optimization, and analytics, pages 113–143. Springer, 2016.
 [LIDLSB11] Manuel LópezIbánez, Jérémie DuboisLacoste, Thomas Stützle, and Mauro Birattari. The irace package, iterated race for automatic algorithm configuration. Technical report, Technical Report TR/IRIDIA/2011004, IRIDIA, Université Libre de Bruxelles, Belgium, 2011.