DeepAI AI Chat
Log In Sign Up

Automatic Differentiation of Programs with Discrete Randomness

by   Gaurav Arya, et al.

Automatic differentiation (AD), a technique for constructing new programs which compute the derivative of an original program, has become ubiquitous throughout scientific computing and deep learning due to the improved performance afforded by gradient-based optimization. However, AD systems have been restricted to the subset of programs that have a continuous dependence on parameters. Programs that have discrete stochastic behaviors governed by distribution parameters, such as flipping a coin with probability p of being heads, pose a challenge to these systems because the connection between the result (heads vs tails) and the parameters (p) is fundamentally discrete. In this paper we develop a new reparameterization-based methodology that allows for generating programs whose expectation is the derivative of the expectation of the original program. We showcase how this method gives an unbiased and low-variance estimator which is as automated as traditional AD mechanisms. We demonstrate unbiased forward-mode AD of discrete-time Markov chains, agent-based models such as Conway's Game of Life, and unbiased reverse-mode AD of a particle filter. Our code package is available at


page 1

page 2

page 3

page 4


ADEV: Sound Automatic Differentiation of Expected Values of Probabilistic Programs

Optimizing the expected values of probabilistic processes is a central p...

Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics

We propose to apply several gradient estimation techniques to enable the...

Automatic Differentiation via Effects and Handlers: An Implementation in Frank

Automatic differentiation (AD) is an important family of algorithms whic...

Differentiating Metropolis-Hastings to Optimize Intractable Densities

We develop an algorithm for automatic differentiation of Metropolis-Hast...

A Brief Introduction to Automatic Differentiation for Machine Learning

Machine learning and neural network models in particular have been impro...

Randomized Automatic Differentiation

The successes of deep learning, variational inference, and many other fi...

Binomial Checkpointing for Arbitrary Programs with No User Annotation

Heretofore, automatic checkpointing at procedure-call boundaries, to red...