Minimizing Regret in Bandit Online Optimization in Unconstrained and Constrained Action Spaces
share
We consider online convex optimization with zeroth-order feedback setting. The decision maker does not know the explicit representation of the time-varying cost functions, or their gradients. At each time step, she observes the value of the current cost function for her chosen action (zeroth-order information). The objective is to minimize the regret, that is, the difference between the sum of the costs she accumulates and that of the optimal action had she known the sequence of cost functions a priori. We present a novel algorithm to minimize regret in both unconstrained and constrained action spaces. Our algorithm hinges on a one-point estimation of the gradients of the cost functions based on their observed values. Moreover, we adapt the presented algorithm to the setting with two-point estimations and demonstrate that the adapted procedure achieves the theoretical lower bound.
READ FULL TEXT