Categorical Distributions of Maximum Entropy under Marginal Constraints

04/07/2022
by   Orestis Loukas, et al.
0

The estimation of categorical distributions under marginal constraints summarizing some sample from a population in the most-generalizable way is key for many machine-learning and data-driven approaches. We provide a parameter-agnostic theoretical framework that enables this task ensuring (i) that a categorical distribution of Maximum Entropy under marginal constraints always exists and (ii) that it is unique. The procedure of iterative proportional fitting (IPF) naturally estimates that distribution from any consistent set of marginal constraints directly in the space of probabilities, thus deductively identifying a least-biased characterization of the population. The theoretical framework together with IPF leads to a holistic workflow that enables modeling any class of categorical distributions solely using the phenomenological information provided.

READ FULL TEXT
research
10/09/2022

The Elliptical Quartic Exponential Distribution: An Annular Distribution Obtained via Maximum Entropy

This paper describes the Elliptical Quartic Exponential distribution in ...
research
02/12/2018

Augment and Reduce: Stochastic Inference for Large Categorical Distributions

Categorical distributions are ubiquitous in machine learning, e.g., in c...
research
06/27/2019

Efficient algorithms for modifying and sampling from a categorical distribution

Probabilistic programming languages and other machine learning applicati...
research
06/27/2022

Entropy-based Characterization of Modeling Constraints

In most data-scientific approaches, the principle of Maximum Entropy (Ma...
research
12/14/2020

Morphology on categorical distributions

The categorical distribution is a natural representation of uncertainty ...
research
03/03/2023

Deep Momentum Multi-Marginal Schrödinger Bridge

Reconstructing population dynamics using only samples from distributions...
research
11/09/2019

Estimation of entropy measures for categorical variables with spatial correlation

Entropy is a measure of heterogeneity widely used in applied sciences, o...

Please sign up or login with your details

Forgot password? Click here to reset