research
∙
06/09/2022
On Neural Architecture Inductive Biases for Relational Tasks
Current deep learning approaches have shown good in-distribution general...
share
research
∙
03/02/2022
Continuous-Time Meta-Learning with Forward Mode Differentiation
Drawing inspiration from gradient-based meta-learning methods with infin...
research
∙
12/28/2020
Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization
The early phase of training has been shown to be important in two ways f...
research
∙
06/22/2020
Advantages of biologically-inspired adaptive neural activation in RNNs during learning
Dynamic adaptation in single-neuron response plays a fundamental role in...
research
∙
06/16/2020
Untangling tradeoffs between recurrence and self-attention in neural networks
Attention and self-attention mechanisms, inspired by cognitive processes...
research
∙
05/28/2019
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics
A recent strategy to circumvent the exploding and vanishing gradient pro...
research
∙
10/06/2018