GO Gradient for Expectation-Based Objectives

by   Yulai Cong, et al.

Within many machine learning algorithms, a fundamental problem concerns efficient calculation of an unbiased gradient wrt parameters for expectation-based objectives _q_ () [f()]. Most existing methods either (i) suffer from high variance, seeking help from (often) complicated variance-reduction techniques; or (ii) they only apply to reparameterizable continuous random variables and employ a reparameterization trick. To address these limitations, we propose a General and One-sample (GO) gradient that (i) applies to many distributions associated with non-reparameterizable continuous or discrete random variables, and (ii) has the same low-variance as the reparameterization trick. We find that the GO gradient often works well in practice based on only one Monte Carlo sample (although one can of course use more samples if desired). Alongside the GO gradient, we develop a means of propagating the chain rule through distributions, yielding statistical back-propagation, coupling neural networks to common random variables.


page 1

page 2

page 3

page 4


The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables

The reparameterization trick enables optimizing large scale stochastic c...

Monte Carlo Estimation of the Density of the Sum of Dependent Random Variables

We introduce a novel unbiased estimator for the density of a sum of rand...

Neural Control Variates for Variance Reduction

In statistics and machine learning, approximation of an intractable inte...

Continuous scaled phase-type distributions

We study random variables arising as the product of phase-type distribut...

Analysis of nested multilevel Monte Carlo using approximate Normal random variables

The multilevel Monte Carlo (MLMC) method has been used for a wide variet...

Generalized Gumbel-Softmax Gradient Estimator for Various Discrete Random Variables

Estimating the gradients of stochastic nodes is one of the crucial resea...

Robust Statistical Comparison of Random Variables with Locally Varying Scale of Measurement

Spaces with locally varying scale of measurement, like multidimensional ...

Please sign up or login with your details

Forgot password? Click here to reset