research
∙
07/10/2023
Dynamics of Temporal Difference Reinforcement Learning
Reinforcement learning has been successful across several applications i...
share
research
∙
07/06/2023
Learning Curves for Heterogeneous Feature-Subsampled Ridge Ensembles
Feature bagging is a well-established ensembling method which aims to re...
research
∙
06/07/2023
Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry
The backpropagation algorithm has experienced remarkable success in trai...
research
∙
06/07/2023
Long Sequence Hopfield Memory
Sequence memory is an essential attribute of natural and artificial inte...
research
∙
04/06/2023
Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks
We analyze the dynamics of finite width effects in wide but finite featu...
research
∙
03/01/2023
Learning curves for deep structured Gaussian feature models
In recent years, significant attention in deep learning theory has been ...
research
∙
01/26/2023
Neural networks learn to magnify areas near decision boundaries
We study how training molds the Riemannian geometry induced by neural ne...
research
∙
12/23/2022
The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes
For small training set sizes P, the generalization error of wide neural ...
research
∙
10/09/2022
Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation
The brain effortlessly extracts latent causes of stimuli, but how it doe...
research
∙
10/05/2022
The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks
It is unclear how changing the learning rule of a deep neural network al...
research
∙
09/27/2022
Biologically-Plausible Determinant Maximization Neural Networks for Blind Separation of Correlated Sources
Extraction of latent sources of complex stimuli is critical for making s...
research
∙
09/21/2022
Interneurons accelerate learning dynamics in recurrent neural networks for statistical adaptation
Early sensory systems in the brain rapidly adapt to fluctuating input st...
research
∙
06/14/2022
Bandwidth Enables Generalization in Quantum Kernel Models
Quantum computers are known to provide speedups over classical state-of-...
research
∙
05/19/2022
Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks
We analyze feature learning in infinite width neural networks trained wi...
research
∙
03/01/2022
Contrasting random and learned features in deep Bayesian linear regression
Understanding how feature learning affects generalization is among the f...
research
∙
01/12/2022
On neural network kernels and the storage capacity problem
In this short note, we reify the connection between work on the storage ...
research
∙
11/23/2021
Depth induces scale-averaging in overparameterized linear Bayesian neural networks
Inference in deep Bayesian neural networks is only fully understood in t...
research
∙
11/10/2021
Attention Approximates Sparse Distributed Memory
While Attention has come to be an important mechanism in deep learning, ...
research
∙
10/29/2021
Neural Networks as Kernel Learners: The Silent Alignment Effect
Neural networks in the lazy training regime converge to kernel machines....
research
∙
10/14/2021
Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
Equivariance has emerged as a desirable property of representations of o...
research
∙
06/04/2021
Learning Curves for SGD on Structured Features
The generalization performance of a machine learning algorithm such as a...
research
∙
06/04/2021
Out-of-Distribution Generalization in Kernel Regression
In real word applications, data generating process for training a machin...
research
∙
06/01/2021
Asymptotics of representation learning in finite Bayesian neural networks
Recent works have suggested that finite Bayesian neural networks may out...
research
∙
04/23/2021
Exact priors of finite neural networks
Bayesian neural networks are theoretically well-understood only in the i...
research
∙
07/21/2020
Activation function dependence of the storage capacity of treelike neural networks
The expressive power of artificial neural networks crucially depends on ...
research
∙
06/30/2020
Associative Memory in Iterated Overparameterized Sigmoid Autoencoders
Recent work showed that overparameterized autoencoders can be trained to...
research
∙
06/23/2020
Statistical Mechanics of Generalization in Kernel Regression
Generalization beyond a training dataset is a main goal of machine learn...
research
∙
04/11/2020
Blind Bounded Source Separation Using Neural Networks with Local Learning Rules
An important problem encountered by both natural and engineered signal p...
research
∙
02/24/2020
Supervised Deep Similarity Matching
We propose a novel biologically-plausible solution to the credit assignm...
research
∙
02/07/2020
Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks
A fundamental question in modern machine learning is how deep neural net...
research
∙
12/11/2019
A Closer Look at Disentangling in β-VAE
In many data analysis tasks, it is beneficial to learn representations w...
research
∙
10/11/2019
Structured and Deep Similarity Matching via Structured and Deep Hebbian Networks
Synaptic plasticity is widely accepted to be the mechanism behind learni...
research
∙
08/05/2019
Neuroscience-inspired online unsupervised learning algorithms
Although the currently popular deep learning networks achieve unpreceden...
research
∙
02/04/2019
A Spiking Neural Network with Local Learning Rules Derived From Nonnegative Similarity Matching
The design and analysis of spiking neural network algorithms will be acc...
research
∙
10/16/2018
Biologically Plausible Online Principal Component Analysis Without Recurrent Neural Dynamics
Artificial neural networks that learn to perform Principal Component Ana...
research
∙
08/06/2018
Efficient Principal Subspace Projection of Streaming Data Through Fast Similarity Matching
Big data problems frequently require processing datasets in a streaming ...
research
∙
06/01/2017
Blind nonnegative source separation using biological neural networks
Blind source separation, i.e. extraction of independent sources from a m...
research
∙
03/23/2017
Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?
Modeling self-organization of neural networks for unsupervised learning ...
research
∙
12/11/2016
Self-calibrating Neural Networks for Dimensionality Reduction
Recently, a novel family of biologically plausible online algorithms for...
research
∙
11/30/2015
Optimization theory of Hebbian/anti-Hebbian networks for PCA and whitening
In analyzing information streamed by sensory organs, our brains face cha...
research
∙
11/30/2015
A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks
To make sense of the world our brains must analyze high-dimensional data...
research
∙
03/02/2015
A Hebbian/Anti-Hebbian Network for Online Sparse Dictionary Learning Derived from Symmetric Matrix Factorization
Olshausen and Field (OF) proposed that neural computations in the primar...
research
∙
03/02/2015
A Hebbian/Anti-Hebbian Network Derived from Online Non-Negative Matrix Factorization Can Cluster and Discover Sparse Features
Despite our extensive knowledge of biophysical properties of neurons, th...
research
∙
03/02/2015
A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data
Neural network models of early sensory processing typically reduce the d...
research
∙
05/12/2014