A Variational Formula for Rényi Divergences

07/07/2020
by   Jeremiah Birrell, et al.
0

We derive a new variational formula for the Rényi family of divergences, R_α(QP), generalizing the classical Donsker-Varadhan variational formula for the Kullback-Leibler divergence. The objective functional in this new variational representation is expressed in terms of expectations under Q and P, and hence can be estimated using samples from the two distributions. We illustrate the utility of such a variational formula by constructing neural-network estimators for the Rényi divergences.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset