On the Kullback-Leibler divergence between location-scale densities
share
We show that the Kullback-Leibler divergence between two densities of potentially different location-scale families can be reduced to the calculation of the Kullback-Leibler divergence between one standard distribution and another location-scale density. In particular, we prove that the Kullback-Leibler divergence between two densities of a scale family depends only on the scale ratio, and report conditions on the standard distribution to get symmetric Kullback-Leibler divergences. We illustrate this symmetric property with the calculation of the Kullback-Leibler divergence between scale Cauchy distributions.
READ FULL TEXT