Nonparametric learning for impulse control problems

09/20/2019
by   Sören Christensen, et al.
0

One of the fundamental assumptions in stochastic control of continuous time processes is that the dynamics of the underlying (diffusion) process is known. This is, however, usually obviously not fulfilled in practice. On the other hand, over the last decades, a rich theory for nonparametric estimation of the drift (and volatility) for continuous time processes has been developed. The aim of this paper is bringing together techniques from stochastic control with methods from statistics for stochastic processes to find a way to both learn the dynamics of the underlying process and control in a reasonable way at the same time. More precisely, we study a long-term average impulse control problem, a stochastic version of the classical Faustmann timber harvesting problem. One of the problems that immediately arises is an exploration vs. exploitation-behavior as is well known for problems in machine learning. We propose a way to deal with this issue by combining exploration- and exploitation periods in a suitable way. Our main finding is that this construction can be based on the rates of convergence of estimators for the invariant density. Using this, we obtain that the average cumulated regret is of uniform order O(T^-1/3).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/17/2022

Estimation and Specification Test for Diffusion Models with Stochastic Volatility

Given the importance of continuous-time stochastic volatility models to ...
research
09/27/2021

Estimating the characteristics of stochastic damping Hamiltonian systems from continuous observations

We consider nonparametric invariant density and drift estimation for a c...
research
03/14/2022

Continuous Time Graph Processes with Known ERGM Equilibria: Contextual Review, Extensions, and Synthesis

Graph processes that unfold in continuous time are of obvious theoretica...
research
06/20/2022

Thompson Sampling Efficiently Learns to Control Diffusion Processes

Diffusion processes that evolve according to linear stochastic different...
research
04/23/2021

Learning to reflect: A unifying approach for data-driven stochastic control strategies

Stochastic optimal control problems have a long tradition in applied pro...
research
08/07/2018

Fluctuation bounds for continuous time branching processes and nonparametric change point detection in growing networks

Motivated by applications, both for modeling real world systems as well ...

Please sign up or login with your details

Forgot password? Click here to reset