A Characterization of Entropy as a Universal Monoidal Natural Transformation

08/10/2023

∙

by   Cheuk Ting Li, et al.

∙

0

∙

share

We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor -/𝖫𝖯𝗋𝗈𝖻_ρ (where 𝖫𝖯𝗋𝗈𝖻_ρ is category of ℓ_ρ discrete probability spaces) to Δ_ℝ. Moreover, the Shannon entropy can be characterized as the universal monoidal natural transformation from -/𝖫𝖯𝗋𝗈𝖻_ρ to the category of strongly Archimedean ordered vector spaces (a reflective subcategory of the lax-slice 2-category over 𝖬𝗈𝗇𝖢𝖺𝗍_ℓ in the 2-category of monoidal categories), providing a succinct characterization of Shannon entropy as a reflection arrow. We can likewise define entropy for every category with a monoidal structure on its under categories (e.g. the category of finite abelian groups, the category of finite inhabited sets, the category of finite dimensional vector spaces, and the augmented simplex category) via the reflection arrow to the reflective subcategory of strongly Archimedean ordered vector spaces. This implies that all these entropies over different categories are components of a single natural transformation (the unit of the idempotent monad), allowing us to connect these entropies in a natural manner. We also provide a universal characterization of the conditional Shannon entropy based on the chain rule which, unlike the characterization of information loss by Baez, Fritz and Leinster, does not require any continuity assumption.