DeepAI
Log In Sign Up

Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems

12/20/2018
by   Dhruv Malik, et al.
0

We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within any pre-specified tolerance of the optimal policy with a number of zero-order evaluations that is an explicit polynomial of the error tolerance, dimension, and curvature properties of the problem. Our analysis reveals some interesting differences between the settings of additive driving noise and random initialization, as well as the settings of one-point and two-point reward feedback. Our theory is corroborated by extensive simulations of derivative-free methods on these systems. Along the way, we derive convergence rates for stochastic zero-order optimization algorithms when applied to a certain class of non-convex problems.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/11/2020

Recent Theoretical Advances in Non-Convex Optimization

Motivated by recent increased interest in optimization algorithms for no...
12/07/2013

Optimal rates for zero-order convex optimization: the power of two function evaluations

We consider derivative-free algorithms for stochastic and non-stochastic...
05/31/2019

Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Games

We study the global convergence of policy optimization for finding the N...
04/11/2018

Derivative free optimization via repeated classification

We develop an algorithm for minimizing a function using n batched functi...
09/11/2012

On the Complexity of Bandit and Derivative-Free Stochastic Convex Optimization

The problem of stochastic convex optimization with bandit feedback (in t...
08/10/2020

Self-accelerating root search and optimisation methods based on rational interpolation

Iteration methods based on barycentric rational interpolation are derive...