DeepAI AI Chat
Log In Sign Up

Neural computation from first principles: Using the maximum entropy method to obtain an optimal bits-per-joule neuron

by   William B Levy, et al.

Optimization results are one method for understanding neural computation from Nature's perspective and for defining the physical limits on neuron-like engineering. Earlier work looks at individual properties or performance criteria and occasionally a combination of two, such as energy and information. Here we make use of Jaynes' maximum entropy method and combine a larger set of constraints, possibly dimensionally distinct, each expressible as an expectation. The method identifies a likelihood-function and a sufficient statistic arising from each such optimization. This likelihood is a first-hitting time distribution in the exponential class. Particular constraint sets are identified that, from an optimal inference perspective, justify earlier neurocomputational models. Interactions between constraints, mediated through the inferred likelihood, restrict constraint-set parameterizations, e.g., the energy-budget limits estimation performance which, in turn, matches an axonal communication constraint. Such linkages are, for biologists, experimental predictions of the method. In addition to the related likelihood, at least one type of constraint set implies marginal distributions, and in this case, a Shannon bits/joule statement arises.


page 1

page 2

page 3

page 4


Observational nonidentifiability, generalized likelihood and free energy

We study the parameter estimation problem in mixture models with observa...

The Elliptical Quartic Exponential Distribution: An Annular Distribution Obtained via Maximum Entropy

This paper describes the Elliptical Quartic Exponential distribution in ...

Maximum Likelihood Constraint Inference from Stochastic Demonstrations

When an expert operates a perilous dynamic system, ideal constraint info...

Filtering Additive Measurement Noise with Maximum Entropy in the Mean

The purpose of this note is to show how the method of maximum entropy in...

Likelihood Ratio Exponential Families

The exponential family is well known in machine learning and statistical...