A short characterization of relative entropy

12/13/2017
by   Tom Leinster, et al.
0

We prove characterization theorems for relative entropy (also known as Kullback-Leibler divergence), q-logarithmic entropy (also known as Tsallis entropy), and q-logarithmic relative entropy. All three have been characterized axiomatically before, but we show that earlier proofs can be simplified considerably, at the same time relaxing some of the hypotheses.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2021

Towards a functorial description of quantum relative entropy

A Bayesian functorial characterization of the classical relative entropy...
research
03/07/2017

Learning opacity in Stratal Maximum Entropy Grammar

Opaque phonological patterns are sometimes claimed to be difficult to le...
research
12/07/2021

On a 2-relative entropy

We construct a 2-categorical extension of the relative entropy functor o...
research
09/18/2021

Sharp Concentration Inequalities for the Centered Relative Entropy

We study the relative entropy between the empirical estimate of a discre...
research
12/02/2019

Adjusted Subadditivity of Relative Entropy for Non-commuting Conditional Expectations

If a set of von Neumann subalgebras has a trivial intersection in finite...
research
11/20/2018

Economics of disagreement -- financial intuition for the Rényi divergence

Lack of accurate intuition is often cited as a scientific challenge, esp...
research
06/12/2023

Analysis of the Relative Entropy Asymmetry in the Regularization of Empirical Risk Minimization

The effect of the relative entropy asymmetry is analyzed in the empirica...

Please sign up or login with your details

Forgot password? Click here to reset