The Stochastic complexity of spin models: How simple are simple spin models?

by   Alberto Beretta, et al.

Simple models, in information theoretic terms, are those with a small stochastic complexity. We study the stochastic complexity of spin models with interactions of arbitrary order. Invariance with respect to bijections within the space of operators allows us to classify models in complexity classes. This invariance also shows that simplicity is not related to the order of the interactions, but rather to their mutual arrangement. Models where statistical dependencies are localized on non-overlapping groups of few variables (and that afford predictions on independencies that are easy to falsify) are simple. On the contrary, fully connected pairwise models, which are often used in statistical learning, are highly complex because of their extended set of interactions.



page 1

page 2

page 3

page 4


Statistical Inference of Minimally Complex Models

Finding the best model that describes a high dimensional dataset, is a d...

Information-theoretical analysis of the statistical dependencies among three variables: Applications to written language

We develop the information-theoretical concepts required to study the st...

Traversal-invariant characterizations of logarithmic space

We give a novel descriptive-complexity theoretic characterization of L a...

Higher-order in-and-outeractions reveal synergy and logical dependence beyond Shannon-information

Information-theoretic quantities reveal dependencies among variables in ...

An information-theoretic approach to hypergraph psychometrics

Psychological network approaches propose to see symptoms or questionnair...

Reconstruction of Pairwise Interactions using Energy-Based Models

Pairwise models like the Ising model or the generalized Potts model have...

On the law of the iterated logarithm and strong invariance principles in stochastic geometry

We study the law of the iterated logarithm (Khinchin (1933), Kolmogorov ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.