Gradient Estimation with Discrete Stein Operators

02/19/2022
by   Jiaxin Shi, et al.
0

Gradient estimation – approximating the gradient of an expectation with respect to the parameters of a distribution – is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize the variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/09/2021

Double Control Variates for Gradient Estimation in Discrete Latent Variable Models

Stochastic gradient-based optimisation for discrete latent variable mode...
research
09/07/2023

DBsurf: A Discrepancy Based Method for Discrete Stochastic Gradient Estimation

Computing gradients of an expectation with respect to the distributional...
research
10/20/2020

VarGrad: A Low-Variance Gradient Estimator for Variational Inference

We analyse the properties of an unbiased gradient estimator of the ELBO ...
research
01/26/2021

Generalized Doubly Reparameterized Gradient Estimators

Efficient low-variance gradient estimation enabled by the reparameteriza...
research
06/15/2022

Training Discrete Deep Generative Models via Gapped Straight-Through Estimator

While deep generative models have succeeded in image processing, natural...
research
10/07/2021

Bias-Variance Tradeoffs in Single-Sample Binary Gradient Estimators

Discrete and especially binary random variables occur in many machine le...
research
10/10/2018

Rao-Blackwellized Stochastic Gradients for Discrete Distributions

We wish to compute the gradient of an expectation over a finite or count...

Please sign up or login with your details

Forgot password? Click here to reset