AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation

by   Jae Hyun Lim, et al.

Entropy is ubiquitous in machine learning, but it is in general intractable to compute the entropy of the distribution of an arbitrary continuous random variable. In this paper, we propose the amortized residual denoising autoencoder (AR-DAE) to approximate the gradient of the log density function, which can be used to estimate the gradient of entropy. Amortization allows us to significantly reduce the error of the gradient approximator by approaching asymptotic optimality of a regular DAE, in which case the estimation is in theory unbiased. We conduct theoretical and experimental analyses on the approximation error of the proposed method, as well as extensive studies on heuristics to ensure its robustness. Finally, using the proposed gradient approximator to estimate the gradient of entropy, we demonstrate state-of-the-art performance on density estimation with variational autoencoders and continuous control with soft actor-critic.


page 5

page 7

page 8

page 17

page 22


Soft Actor-Critic with Cross-Entropy Policy Optimization

Soft Actor-Critic (SAC) is one of the state-of-the-art off-policy reinfo...

Actor-Critic learning for mean-field control in continuous time

We study policy gradient for mean-field control in continuous time in a ...

Density Estimation using Entropy Maximization for Semi-continuous Data

Semi-continuous data comes from a distribution that is a mixture of the ...

Stochastic Convergence Results for Regularized Actor-Critic Methods

In this paper, we present a stochastic convergence proof, under suitable...

Regularized Multi-Task Learning for Multi-Dimensional Log-Density Gradient Estimation

Log-density gradient estimation is a fundamental statistical problem and...

Gradient density estimation in arbitrary finite dimensions using the method of stationary phase

We prove that the density function of the gradient of a sufficiently smo...

On Tsallis Entropy Bias and Generalized Maximum Entropy Models

In density estimation task, maximum entropy model (Maxent) can effective...

Code Repositories


AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation

view repo


AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation

view repo

Please sign up or login with your details

Forgot password? Click here to reset