An information scaling law: ζ= 3/4

10/25/2017
by   Michael C. Abbott, et al.
0

Consider the entropy of a unit Gaussian convolved over a discrete set of K points, constrained to an interval of length L. Maximising this entropy fixes K, and we show that this number exhibits a novel scaling law K L^1/ζ as L -> infinity, with exponent ζ = 3/4. This law was observed numerically in a recent paper about optimal effective theories; here we present an analytic derivation. We argue that this law is generic for channel capacity maximisation, or the equivalent minimax problem. We also briefly discuss the behaviour at the boundary of the interval, and higher dimensional versions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/08/2017

A Simple Derivation of the Heap's Law from the Generalized Zipf's Law

I reproduce a rather simple formal derivation of the Heaps' law from the...
research
04/23/2019

A Refined Scaling Law for Spatially Coupled LDPC Codes Over the Binary Erasure Channel

We propose a refined scaling law to predict the finite-length performanc...
research
03/23/2023

The Quantization Model of Neural Scaling

We propose the Quantization Model of neural scaling laws, explaining bot...
research
02/27/2014

Scaling hypothesis for the Euclidean bipartite matching problem

We propose a simple yet very predictive form, based on a Poisson's equat...
research
01/31/2023

Scaling laws for single-agent reinforcement learning

Recent work has shown that, in generative modeling, cross-entropy loss i...
research
02/27/2020

Scaling exponents saturate in three-dimensional isotropic turbulence

From a database of direct numerical simulations of homogeneous and isotr...
research
10/09/2020

Generalization of the power-law rating curve using hydrodynamic theory and Bayesian hierarchical modeling

The power-law rating curve has been used extensively in hydraulic practi...

Please sign up or login with your details

Forgot password? Click here to reset