research
∙
09/20/2023
The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute
The Languini Kitchen serves as both a research collective and codebase d...
share
research
∙
06/30/2023
The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit
In deep learning theory, the covariance matrix of the representations se...
research
∙
06/23/2023
Scaling MLPs: A Tale of Inductive Bias
In this work we revisit the most fundamental building block in deep lear...
research
∙
06/04/2023
Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes
Training models to apply common-sense linguistic knowledge and visual co...
research
∙
05/16/2023
The Hessian perspective into the Nature of Convolutional Neural Networks
While Convolutional Neural Networks (CNNs) have long been investigated a...
research
∙
04/12/2023
CLIP-Guided Vision-Language Pre-training for Question Answering in 3D Scenes
Training models to apply linguistic knowledge and visual concepts from 2...
research
∙
03/16/2023
Achieving a Better Stability-Plasticity Trade-off via Auxiliary Networks in Continual Learning
In contrast to the natural capabilities of humans to learn new tasks in ...
research
∙
02/23/2023
Random Teachers are Good Teachers
In this work, we investigate the implicit regularization induced by teac...
research
∙
11/22/2022
Cosmology from Galaxy Redshift Surveys with PointNet
In recent years, deep learning approaches have achieved state-of-the-art...
research
∙
10/25/2022
The Curious Case of Benign Memorization
Despite the empirical advances of deep learning across a variety of lear...
research
∙
10/21/2022
Decoding a Neural Retriever's Latent Space for Query Suggestion
Neural retrieval models have superseded classic bag-of-words methods suc...
research
∙
10/03/2022
Mastering Spatial Graph Prediction of Road Networks
Accurately predicting road networks from satellite images requires a glo...
research
∙
07/19/2022
OpenFilter: A Framework to Democratize Research Access to Social Media AR Filters
Augmented Reality or AR filters on selfies have become very popular on s...
research
∙
05/27/2022
How Tempering Fixes Data Augmentation in Bayesian Neural Networks
While Bayesian neural networks (BNNs) provide a sound and principled alt...
research
∙
03/14/2022
Phenomenology of Double Descent in Finite-Width Neural Networks
`Double descent' delineates the generalization behaviour of models depen...
research
∙
03/07/2022
Generalization Through The Lens Of Leave-One-Out Error
Despite the tremendous empirical success of deep learning models to solv...
research
∙
01/26/2022
FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control
Generating music with deep neural networks has been an area of active re...
research
∙
01/02/2022
Randomized Signature Layers for Signal Extraction in Time Series Data
Time series analysis is a widespread task in Natural Sciences, Social Sc...
research
∙
09/01/2021
Boosting Search Engines with Interactive Agents
Can machines learn to use a search engine as an interactive tool for fin...
research
∙
08/04/2021
How to Query Language Models?
Large pre-trained language models (LMs) are capable of not only recoveri...
research
∙
06/30/2021
Analytic Insights into Structure and Rank of Neural Network Hessian Maps
The Hessian of a neural network captures parameter interactions through ...
research
∙
06/11/2021
Precise characterization of the prior predictive distribution of deep ReLU networks
Recent works on Bayesian neural networks (BNNs) have highlighted the nee...
research
∙
06/11/2021
Disentangling the Roles of Curation, Data-Augmentation and the Prior in the Cold Posterior Effect
The "cold posterior effect" (CPE) in Bayesian deep learning describes th...
research
∙
05/07/2021
Uniform Convergence, Adversarial Spheres and a Simple Remedy
Previous work has cast doubt on the general framework of uniform converg...
research
∙
03/23/2021
Generative Minimization Networks: Training GANs Without Competition
Many applications in machine learning can be framed as minimization prob...
research
∙
03/21/2021
SwissDial: Parallel Multidialectal Corpus of Spoken Swiss German
Swiss German is a dialect continuum whose natively acquired dialects sig...
research
∙
02/23/2021
Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization
Viewing optimization methods as numerical integrators for ordinary diffe...
research
∙
06/13/2020
Convolutional Generation of Textured 3D Meshes
Recent generative models for 2D images achieve impressive visual results...
research
∙
03/05/2020
BERT as a Teacher: Contextual Embeddings for Sequence-Level Reward
Measuring the quality of a generated sequence against a set of reference...
research
∙
12/06/2019
Controlling Style and Semantics in Weakly-Supervised Image Generation
We propose a weakly-supervised approach for conditional image generation...
research
∙
10/31/2019
Mixing of Stochastic Accelerated Gradient Descent
We study the mixing properties for stochastic accelerated gradient desce...
research
∙
09/04/2019
LeDeepChef: Deep Reinforcement Learning Agent for Families of Text-Based Games
While Reinforcement Learning (RL) approaches lead to significant achieve...
research
∙
08/30/2019
Autoregressive Text Generation Beyond Feedback Loops
Autoregressive state transitions, where predictions are conditioned on p...
research
∙
08/15/2019
Cosmological N-body simulations: a challenge for scalable generative models
Deep generative models, such as Generative Adversarial Networks (GANs) o...
research
∙
06/04/2019
Adversarial Training Generalizes Data-dependent Spectral Norm Regularization
We establish a theoretical link between adversarial training and operato...
research
∙
02/13/2019
The Odds are Odd: A Statistical Test for Detecting Adversarial Examples
We investigate conditions under which test statistics exist that can rel...
research
∙
11/13/2018
Evaluating GANs via Duality
Generative Adversarial Networks (GANs) have shown great results in accur...
research
∙
09/23/2018
Learning and Evaluating Sparse Interpretable Sentence Embeddings
Previous research on word embeddings has shown that sparse representatio...
research
∙
08/23/2018
End-to-End Neural Entity Linking
Entity Linking (EL) is an essential task for semantic text understanding...
research
∙
06/20/2018
A Distributed Second-Order Algorithm You Can Trust
Due to the rapid growth of data and computational resources, distributed...
research
∙
06/12/2018
Deep State Space Models for Unconditional Word Generation
Autoregressive feedback is considered a necessity for successful uncondi...
research
∙
05/27/2018
Towards a Theoretical Understanding of Batch Normalization
Normalization techniques such as Batch Normalization have been applied v...
research
∙
05/25/2018
Zero-Shot Dual Machine Translation
Neural Machine Translation (NMT) systems rely on large amounts of parall...
research
∙
05/23/2018
Hyperbolic Neural Networks
Hyperbolic spaces have recently gained momentum in the context of machin...
research
∙
05/22/2018
Adversarially Robust Training through Structured Gradient Regularization
We propose a novel data-dependent structured gradient regularizer to inc...
research
∙
05/15/2018
Local Saddle Point Optimization: A Curvature Exploitation Approach
Gradient-based optimization methods are the most popular choice for find...
research
∙
04/03/2018
Hyperbolic Entailment Cones for Learning Hierarchical Embeddings
Learning graph representations via low-dimensional embeddings that prese...
research
∙
03/15/2018
Escaping Saddles with Stochastic Gradients
We analyze the variance of stochastic gradients along negative curvature...
research
∙
01/27/2018
Fast Cosmic Web Simulations with Generative Adversarial Networks
Dark matter in the universe evolves through gravity to form a complex ne...
research
∙
11/15/2017