On the Rényi Cross-Entropy

06/28/2022
by   Ferenc Cole Thierrin, et al.
0

The Rényi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we examine the properties of this measure and derive closed-form expressions for it when one of the distributions is fixed and when both distributions belong to the exponential family. We also analytically determine a formula for the cross-entropy rate for stationary Gaussian processes and for finite-alphabet Markov sources.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset