On the Impossibility of Statistically Improving Empirical Optimization: A Second-Order Stochastic Dominance Perspective

05/27/2021
by   Henry Lam, et al.
0

When the underlying probability distribution in a stochastic optimization is observed only through data, various data-driven formulations have been studied to obtain approximate optimal solutions. We show that no such formulations can, in a sense, theoretically improve the statistical quality of the solution obtained from empirical optimization. We argue this by proving that the first-order behavior of the optimality gap against the oracle best solution, which includes both the bias and variance, for any data-driven solution is second-order stochastically dominated by empirical optimization, as long as suitable smoothness holds with respect to the underlying distribution. We demonstrate this impossibility of improvement in a range of examples including regularized optimization, distributionally robust optimization, parametric optimization and Bayesian generalizations. We also discuss the connections of our results to semiparametric statistical inference and other perspectives in the data-driven optimization literature.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

06/23/2021

Bayesian Joint Chance Constrained Optimization: Approximations and Statistical Consistency

This paper considers data-driven chance-constrained stochastic optimizat...
10/05/2018

Bounding Optimality Gap in Stochastic Optimization via Bagging: Statistical Efficiency and Stability

We study a statistical method to estimate the optimal value, and the opt...
10/11/2016

Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach

We study statistical inference and robust solution methods for stochasti...
01/04/2022

Stability and Sample-based Approximations of Composite Stochastic Optimization Problems

Optimization under uncertainty and risk is indispensable in many practic...
01/08/2021

Heteroscedasticity-aware residuals-based contextual stochastic optimization

We explore generalizations of some integrated learning and optimization ...
09/14/2021

Learning and Decision-Making with Data: Optimal Formulations and Phase Transitions

We study the problem of designing optimal learning and decision-making f...
04/23/2021

Learning to reflect: A unifying approach for data-driven stochastic control strategies

Stochastic optimal control problems have a long tradition in applied pro...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.