Divergence functions in dually flat spaces and their properties
share
Dually flat spaces play a key role in the differential geometrical approach in statistics (information geometry) and many divergences have been studied as an amount which measures the discrepancy between two probability distributions. In a dually flat space, there exist dual affine coordinate systems and convex functions called potential and a canonical divergence is naturally introduced as the function of the affine coordinates and potentials.The canonical divergence satisfies a relational expression called triangular relation. This can be regarded as a generalization of the law of cosines in Euclidean space. In this paper, we newly introduce two kinds of divergences. The first divergence is a function of affine coordinates and it is consistent with the Jeffreys divergence for exponential or mixture families. For this divergence, we show that more relational equations and theorems similar to Euclidean space hold in addition to the law of cosines. The second divergences are functions of potentials and they are consistent with the Bhattacharyya distance for exponential families and are consistent with the Jensen-Shannon divergence for mixture families respectively. We derive an inequality between the the first and the second divergences and show that the inequality is a generalization of Lin's inequality.
READ FULL TEXT