DeepAI
Log In Sign Up

Learning to Stop with Surprisingly Few Samples

02/19/2021
by   Daniel Russo, et al.
0

We consider a discounted infinite horizon optimal stopping problem. If the underlying distribution is known a priori, the solution of this problem is obtained via dynamic programming (DP) and is given by a well known threshold rule. When information on this distribution is lacking, a natural (though naive) approach is "explore-then-exploit," whereby the unknown distribution or its parameters are estimated over an initial exploration phase, and this estimate is then used in the DP to determine actions over the residual exploitation phase. We show: (i) with proper tuning, this approach leads to performance comparable to the full information DP solution; and (ii) despite common wisdom on the sensitivity of such "plug in" approaches in DP due to propagation of estimation errors, a surprisingly "short" (logarithmic in the horizon) exploration horizon suffices to obtain said performance. In cases where the underlying distribution is heavy-tailed, these observations are even more pronounced: a single sample exploration phase suffices.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/28/2021

Dynamic Programming and Linear Programming for Odds Problem

This paper discusses the odds problem, proposed by Bruss in 2000, and it...
11/07/2022

Discrete Distribution Estimation under User-level Local Differential Privacy

We study discrete distribution estimation under user-level local differe...
11/04/2019

Lookahead Bayesian Optimization via Rollout: Guarantees and Sequential Rolling Horizons

Lookahead, also known as non-myopic, Bayesian optimization (BO) aims to ...
03/21/2020

DP-Net: Dynamic Programming Guided Deep Neural Network Compression

In this work, we propose an effective scheme (called DP-Net) for compres...
08/16/2022

Private Estimation with Public Data

We initiate the study of differentially private (DP) estimation with acc...
06/11/2019

Macro-action Multi-timescale Dynamic Programming for Energy Management with Phase Change Materials

This paper focuses on home energy management systems (HEMS) in buildings...