On the Kullback-Leibler divergence between location-scale densities

04/23/2019

∙

by   Frank Nielsen, et al.

∙

0

∙

share

We show that the Kullback-Leibler divergence between two densities of potentially different location-scale families can be reduced to the calculation of the Kullback-Leibler divergence between one standard distribution and another location-scale density. In particular, we prove that the Kullback-Leibler divergence between two densities of a scale family depends only on the scale ratio, and report conditions on the standard distribution to get symmetric Kullback-Leibler divergences. We illustrate this symmetric property with the calculation of the Kullback-Leibler divergence between scale Cauchy distributions.