Disentangling with Biological Constraints: A Theory of Functional Cell Types

09/30/2022
by   James C. R. Whittington, et al.
0

Neurons in the brain are often finely tuned for specific task variables. Moreover, such disentangled representations are highly sought after in machine learning. Here we mathematically prove that simple biological constraints on neurons, namely nonnegativity and energy efficiency in both activity and weights, promote such sought after disentangled representations by enforcing neurons to become selective for single factors of task variation. We demonstrate these constraints lead to disentangling in a variety of tasks and architectures, including variational autoencoders. We also use this theory to explain why the brain partitions its cells into distinct cell types such as grid and object-vector cells, and also explain when the brain instead entangles representations in response to entangled task factors. Overall, this work provides a mathematical understanding of why, when, and how neurons represent factors in both brains and machines, and is a first step towards understanding of how task demands structure neural representations.

READ FULL TEXT

page 7

page 8

page 28

page 31

research
06/01/2023

Neuronal Cell Type Classification using Deep Learning

The brain is likely the most complex organ, given the variety of functio...
research
11/03/2021

Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity

Meaningful and simplified representations of neural activity can yield i...
research
06/13/2018

Cell Identity Codes: Understanding Cell Identity from Gene Expression Profiles using Deep Neural Networks

Understanding cell identity is an important task in many biomedical area...
research
03/28/2020

Objective Multi-variable Classification and Inference of Biological Neuronal Networks

Classification of biological neuron types and networks poses challenges ...
research
07/03/2019

Multiplicative modulations in hue-selective cells enhance unique hue representation

There is still much to understand about the color processing mechanisms ...
research
11/01/2021

Deep neural networks as nested dynamical systems

There is an analogy that is often made between deep neural networks and ...
research
07/01/2019

DeepTEGINN: Deep Learning Based Tools to Extract Graphs from Images of Neural Networks

In the brain, the structure of a network of neurons defines how these ne...

Please sign up or login with your details

Forgot password? Click here to reset