research
∙
05/30/2023
Synaptic Weight Distributions Depend on the Geometry of Plasticity
Most learning algorithms in machine learning rely on gradient descent to...
share
research
∙
02/24/2023
Flexible Phase Dynamics for Bio-Plausible Contrastive Learning
Many learning algorithms used as normative models in neuroscience or as ...
research
∙
02/22/2023
Steerable Equivariant Representation Learning
Pre-trained deep image representations are useful for post-training task...
research
∙
02/13/2023
Sources of Richness and Ineffability for Phenomenally Conscious States
Conscious states (states that there is something it is like to be in) se...
research
∙
11/29/2022
Transfer Entropy Bottleneck: Learning Sequence to Sequence Information Transfer
When presented with a data stream of two statistically dependent variabl...
research
∙
10/28/2022
Reliability of CKA as a Similarity Measure in Deep Learning
Comparing learned neural representations in neural networks is a challen...
research
∙
10/25/2022
From Points to Functions: Infinite-dimensional Representations in Diffusion Models
Diffusion-based generative models learn to iteratively transfer unstruct...
research
∙
09/19/2022
Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty
Among attempts at giving a theoretical account of the success of deep ne...
research
∙
06/09/2022
On Neural Architecture Inductive Biases for Relational Tasks
Current deep learning approaches have shown good in-distribution general...
research
∙
06/06/2022
Is a Modular Architecture Enough?
Inspired from human cognition, machine learning systems are gradually re...
research
∙
06/02/2022
Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules
To unveil how the brain learns, ongoing work seeks biologically-plausibl...
research
∙
03/02/2022
Continuous-Time Meta-Learning with Forward Mode Differentiation
Drawing inspiration from gradient-based meta-learning methods with infin...
research
∙
02/24/2022
Clarifying MCMC-based training of modern EBMs : Contrastive Divergence versus Maximum Likelihood
The Energy-Based Model (EBM) framework is a very general approach to gen...
research
∙
12/06/2021
Multi-scale Feature Learning Dynamics: Insights for Double Descent
A key challenge in building theoretical foundations for deep learning is...
research
∙
10/18/2021
Compositional Attention: Disentangling Search and Retrieval
Multi-head, key-value attention is the backbone of the widely successful...
research
∙
07/26/2021
Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance
In modern relational machine learning it is common to encounter large gr...
research
∙
07/02/2021
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Inducing causal relationships from observations is a classic problem in ...
research
∙
05/28/2021
Efficient and robust multi-task learning in the brain with modular task primitives
In a real-world setting biological agents do not have infinite resources...
research
∙
11/18/2020
Gradient Starvation: A Learning Proclivity in Neural Networks
We identify and formalize a fundamental gradient descent phenomenon resu...
research
∙
10/26/2020
LEAD: Least-Action Dynamics for Min-Max Optimization
Adversarial formulations such as generative adversarial networks (GANs) ...
research
∙
08/03/2020
Implicit Regularization in Deep Learning: A View from Function Space
We approach the problem of implicit regularization in deep learning from...
research
∙
06/30/2020
Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules
Robust perception relies on both bottom-up and top-down signals. Bottom-...
research
∙
06/25/2020
On Lyapunov Exponents for RNNs: Understanding Information Propagation Using Dynamical Systems Tools
Recurrent neural networks (RNNs) have been successfully applied to a var...
research
∙
06/22/2020
Advantages of biologically-inspired adaptive neural activation in RNNs during learning
Dynamic adaptation in single-neuron response plays a fundamental role in...
research
∙
06/16/2020
Untangling tradeoffs between recurrence and self-attention in neural networks
Attention and self-attention mechanisms, inspired by cognitive processes...
research
∙
01/09/2020
Internal representation dynamics and geometry in recurrent neural networks
The efficiency of recurrent neural networks (RNNs) in dealing with seque...
research
∙
06/02/2019
Dimensionality compression and expansion in Deep Neural Networks
Datasets such as images, text, or movies are embedded in high-dimensiona...
research
∙
05/28/2019