Log In Sign Up

Contrastive Topographic Models: Energy-based density models applied to the understanding of sensory coding and cortical topography

by   Simon Osindero, et al.

We address the problem of building theoretical models that help elucidate the function of the visual brain at computational/algorithmic and structural/mechanistic levels. We seek to understand how the receptive fields and topographic maps found in visual cortical areas relate to underlying computational desiderata. We view the development of sensory systems from the popular perspective of probability density estimation; this is motivated by the notion that an effective internal representational scheme is likely to reflect the statistical structure of the environment in which an organism lives. We apply biologically based constraints on elements of the model. The thesis begins by surveying the relevant literature from the fields of neurobiology, theoretical neuroscience, and machine learning. After this review we present our main theoretical and algorithmic developments: we propose a class of probabilistic models, which we refer to as "energy-based models", and show equivalences between this framework and various other types of probabilistic model such as Markov random fields and factor graphs; we also develop and discuss approximate algorithms for performing maximum likelihood learning and inference in our energy based models. The rest of the thesis is then concerned with exploring specific instantiations of such models. By performing constrained optimisation of model parameters to maximise the likelihood of appropriate, naturalistic datasets we are able to qualitatively reproduce many of the receptive field and map properties found in vivo, whilst simultaneously learning about statistical regularities in the data.


Neural Density Estimation and Likelihood-free Inference

I consider two problems in machine learning and statistics: the problem ...

Statistical applications of contrastive learning

The likelihood function plays a crucial role in statistical inference an...

Normative theory of visual receptive fields

This article gives an overview of a normative computational theory of vi...

Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP

Structured prediction in natural language processing (NLP) has a long hi...

Reproducing sensory induced hallucinations via neural fields

Understanding sensory-induced cortical patterns in the primary visual co...

FRAME Revisited: An Interpretation View Based on Particle Evolution

FRAME (Filters, Random fields, And Maximum Entropy) is an energy-based d...