DeepAI AI Chat
Log In Sign Up

A Variational Formula for Rényi Divergences

by   Jeremiah Birrell, et al.
University of Massachusetts Amherst
Brown University

We derive a new variational formula for the Rényi family of divergences, R_α(QP), generalizing the classical Donsker-Varadhan variational formula for the Kullback-Leibler divergence. The objective functional in this new variational representation is expressed in terms of expectations under Q and P, and hence can be estimated using samples from the two distributions. We illustrate the utility of such a variational formula by constructing neural-network estimators for the Rényi divergences.


page 1

page 2

page 3

page 4


A note on some information-theoretic divergences between Zeta distributions

In this short communication, we first report a closed-form formula for c...

Function-space regularized Rényi divergences

We propose a new family of regularized Rényi divergences parametrized no...

Rediscovery of Numerical Lüscher's Formula from the Neural Network

We present that by predicting the spectrum in discrete space from the ph...

Tighter Variational Representations of f-Divergences via Restriction to Probability Measures

We show that the variational representations for f-divergences currently...

Improved GM(1,1) model based on Simpson formula and its applications

The classical GM(1,1) model is an efficient tool to make accurate foreca...

Variable-Length Intrinsic Randomness Allowing Positive Value of the Average Variational Distance

This paper considers the problem of variable-length intrinsic randomness...

The Canny-Emiris conjecture for the sparse resultant

We present a product formula for the initial parts of the sparse resulta...