Stochastic Implicit Natural Gradient for Black-box Optimization

by   Yueming Lyu, et al.

Black-box optimization is primarily important for many compute-intensive applications, including reinforcement learning (RL), robot control, etc. This paper presents a novel theoretical framework for black-box optimization, in which our method performs stochastic update within a trust region defined with KL-divergence. We show that this update is equivalent to a natural gradient step w.r.t. natural parameters of an exponential-family distribution. Theoretically, we prove the convergence rate of our framework for convex functions. Our theoretical results also hold for non-differentiable black-box functions. Empirically, our method achieves superior performance compared with the state-of-the-art method CMA-ES on separable benchmark test problems.


page 1

page 2

page 3

page 4


SABLAS: Learning Safe Control for Black-box Dynamical Systems

Control certificates based on barrier functions have been a powerful too...

Benchmarking the Hooke-Jeeves Method, MTS-LS1, and BSrr on the Large-scale BBOB Function Set

This paper investigates the performance of three black-box optimizers ex...

Natural Evolution Strategies

This paper presents Natural Evolution Strategies (NES), a recent family ...

Regret-Aware Black-Box Optimization with Natural Gradients, Trust-Regions and Entropy Control

Most successful stochastic black-box optimizers, such as CMA-ES, use ran...

Fourier Representations for Black-Box Optimization over Categorical Variables

Optimization of real-world black-box functions defined over purely categ...

Asymptotic convergence rates for averaging strategies

Parallel black box optimization consists in estimating the optimum of a ...

AdaDGS: An adaptive black-box optimization method with a nonlocal directional Gaussian smoothing gradient

The local gradient points to the direction of the steepest slope in an i...