Deep Data Density Estimation through Donsker-Varadhan Representation

04/14/2021
by   Seonho Park, et al.
0

Estimating the data density is one of the challenging problems in deep learning. In this paper, we present a simple yet effective method for estimating the data density using a deep neural network and the Donsker-Varadhan variational lower bound on the KL divergence. We show that the optimal critic function associated with the Donsker-Varadhan representation on the KL divergence between the data and the uniform distribution can estimate the data density. We also present the deep neural network-based modeling and its stochastic learning. The experimental results and possible applications of the proposed method demonstrate that it is competitive with the previous methods and has a lot of possibilities in applied to various applications.

READ FULL TEXT

page 2

page 6

research
09/14/2018

Variational Autoencoder with Implicit Optimal Priors

The variational autoencoder (VAE) is a powerful generative model that ca...
research
09/29/2021

Reliable Estimation of KL Divergence using a Discriminator in Reproducing Kernel Hilbert Space

Estimating Kullback Leibler (KL) divergence from samples of two distribu...
research
11/09/2017

A Separation Principle for Control in the Age of Deep Learning

We review the problem of defining and inferring a "state" for a control ...
research
05/10/2021

Neural Computation of Capacity Region of Memoryless Multiple Access Channels

This paper provides a numerical framework for computing the achievable r...
research
09/09/2021

Stationary Density Estimation of Itô Diffusions Using Deep Learning

In this paper, we consider the density estimation problem associated wit...
research
06/30/2014

Direct Density-Derivative Estimation and Its Application in KL-Divergence Approximation

Estimation of density derivatives is a versatile tool in statistical dat...
research
09/01/2020

Stochastic Graph Recurrent Neural Network

Representation learning over graph structure data has been widely studie...

Please sign up or login with your details

Forgot password? Click here to reset