Learning in Integer Latent Variable Models with Nested Automatic Differentiation

06/08/2018
by   Daniel Sheldon, et al.
0

We develop nested automatic differentiation (AD) algorithms for exact inference and learning in integer latent variable models. Recently, Winner, Sujono, and Sheldon showed how to reduce marginalization in a class of integer latent variable models to evaluating a probability generating function which contains many levels of nested high-order derivatives. We contribute faster and more stable AD algorithms for this challenging problem and a novel algorithm to compute exact gradients for learning. These contributions lead to significantly faster and more accurate learning algorithms, and are the first AD algorithms whose running time is polynomial in the number of levels of nesting.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/17/2018

A Tutorial on Deep Latent Variable Models of Natural Language

There has been much recent, exciting work on combining the complementary...
research
05/13/2023

Automatic Differentiation in Prolog

Automatic differentiation (AD) is a range of algorithms to compute the n...
research
11/26/2019

Recursive multivariate derivatives of e^f(x_1,..., x_n) of arbitrary order

High-order derivatives of nested functions of a single variable can be c...
research
09/22/2022

Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training

An activation function is an element-wise mathematical function and play...
research
02/13/2018

Leveraging the Exact Likelihood of Deep Latent Variable Models

Deep latent variable models combine the approximation abilities of deep ...
research
09/09/2021

Neural Latents Benchmark '21: Evaluating latent variable models of neural population activity

Advances in neural recording present increasing opportunities to study n...
research
05/17/2022

Latent Variable Method Demonstrator – Software for Understanding Multivariate Data Analytics Algorithms

The ever-increasing quantity of multivariate process data is driving a n...

Please sign up or login with your details

Forgot password? Click here to reset