DeepAI AI Chat
Log In Sign Up

MuProp: Unbiased Backpropagation for Stochastic Neural Networks

11/16/2015
by   Shixiang Gu, et al.
Google
0

Deep neural networks are powerful parametric models that can be trained efficiently using the backpropagation algorithm. Stochastic neural networks combine the power of large parametric functions with that of graphical models, which makes it possible to learn very complex distributions. However, as backpropagation is not directly applicable to stochastic networks that include discrete sampling operations within their computational graph, training such networks remains difficult. We present MuProp, an unbiased gradient estimator for stochastic networks, designed to make this task easier. MuProp improves on the likelihood-ratio estimator by reducing its variance using a control variate based on the first-order Taylor expansion of a mean-field network. Crucially, unlike prior attempts at using backpropagation for training stochastic networks, the resulting estimator is unbiased and well behaved. Our experiments on structured output prediction and discrete latent variable modeling demonstrate that MuProp yields consistently good performance across a range of difficult tasks.

READ FULL TEXT
10/26/2021

CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator

Accurately backpropagating the gradient through categorical variables is...
10/14/2021

Hindsight Network Credit Assignment: Efficient Credit Assignment in Networks of Discrete Stochastic Units

Training neural networks with discrete stochastic variables presents a u...
11/24/2020

Hindsight Network Credit Assignment

We present Hindsight Network Credit Assignment (HNCA), a novel learning ...
04/01/2020

SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models

Standard variational lower bounds used to train latent variable models p...
07/25/2018

Backprop-Q: Generalized Backpropagation for Stochastic Computation Graphs

In real-world scenarios, it is appealing to learn a model carrying out s...
11/04/2016

Reparameterization trick for discrete variables

Low-variance gradient estimation is crucial for learning directed graphi...
07/28/2015

Training recurrent networks online without backtracking

We introduce the "NoBackTrack" algorithm to train the parameters of dyna...