Curvature of Feasible Sets in Offline and Online Optimization
share
It is known that the curvature of the feasible set in convex optimization allows for algorithms with better convergence rates, and there has been renewed interest in this topic both for offline as well as online problems. In this paper, leveraging results on geometry and convex analysis, we further our understanding of the role of curvature in optimization: - We first show the equivalence of two notions of curvature, namely strong convexity and gauge bodies, proving a conjecture of Abernethy et al. As a consequence, this show that the Frank-Wolfe-type method of Wang and Abernethy has accelerated convergence rate O(1/t^2) over strongly convex feasible sets without additional assumptions on the (convex) objective function. - In Online Linear Optimization, we show that the logarithmic regret guarantee of the algorithm Follow the Leader (FTL) over strongly convex sets recently proved by Huang et al. follows directly from a partial lipschitzness of the support function of such sets. We believe that this provides a simpler explanation for the good performance of FTL in this context. - We provide an efficient procedure for approximating convex bodies by curved ones, smoothly trading off approximation error and curvature, allowing one to extend the applicability of algorithms for curved set to non-curved ones.
READ FULL TEXT