research
∙
06/01/2023
Faster Causal Attention Over Large Sequences Through Sparse Flash Attention
Transformer-based language models have found many diverse applications r...
share
research
∙
04/21/2023
SequeL: A Continual Learning Library in PyTorch and JAX
Continual Learning is an important and challenging problem in machine le...
research
∙
02/10/2023
Graph Neural Networks Go Forward-Forward
We present the Graph Forward-Forward (GFF) algorithm, an extension of th...
research
∙
01/18/2023
Learning Deformation Trajectories of Boltzmann Densities
We introduce a training objective for continuous normalizing flows that ...
research
∙
11/21/2022
ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields
We present ESLAM, an efficient implicit neural representation method for...
research
∙
10/25/2022
Deformation Theory of Boltzmann Distributions
Consider a one-parameter family of Boltzmann distributions p_t(x) = 1Z_t...
research
∙
10/20/2022
Accurate Extrinsic Prediction of Physical Systems Using Transformers
Accurate high-altitude wind forecasting is important for air traffic con...
research
∙
10/18/2022
Pareto Manifold Learning: Tackling multiple tasks via ensembles of single-task models
In Multi-Task Learning, tasks may compete and limit the performance achi...
research
∙
09/01/2022
Transformers are Sample Efficient World Models
Deep reinforcement learning agents are notoriously sample inefficient, w...
research
∙
06/14/2022
Flatten the Curve: Efficiently Training Low-Curvature Neural Networks
The highly non-linear nature of deep neural networks causes them to be s...
research
∙
05/30/2022
Flowification: Everything is a Normalizing Flow
We develop a method that can be used to turn any multi-layer perceptron ...
research
∙
03/07/2022
HyperMixer: An MLP-based Green AI Alternative to Transformers
Transformer-based architectures are the model of choice for natural lang...
research
∙
03/02/2022
The Theoretical Expressiveness of Maxpooling
Over the decade since deep neural networks became state of the art image...
research
∙
02/17/2022
MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned
Reinforcement learning competitions advance the field by providing appro...
research
∙
02/11/2022
Borrowing from yourself: Faster future video segmentation with partial channel update
Semantic segmentation is a well-addressed topic in the computer vision l...
research
∙
02/10/2022
SUPA: A Lightweight Diagnostic Simulator for Machine Learning in Particle Physics
Deep learning methods have gained popularity in high energy physics for ...
research
∙
02/09/2022
Agree to Disagree: Diversity through Disagreement for Better Transferability
Gradient-based learning algorithms have an implicit simplicity bias whic...
research
∙
12/20/2021
Efficient Wind Speed Nowcasting with GPU-Accelerated Nearest Neighbors Algorithm
This paper proposes a simple yet efficient high-altitude wind nowcasting...
research
∙
11/26/2021
GeoNeRF: Generalizing NeRF with Geometry Priors
We present GeoNeRF, a generalizable photorealistic novel view synthesis ...
research
∙
10/19/2021
Test time Adaptation through Perturbation Robustness
Data samples generated by several real world processes are dynamic in na...
research
∙
09/08/2021
Priming PCA with EigenGame
We introduce primed-PCA (pPCA), an extension of the recently proposed Ei...
research
∙
04/16/2021
Language Models are Few-Shot Butlers
Pretrained language models demonstrate strong performance in most NLP ta...
research
∙
04/13/2021
Structural analysis of an all-purpose question answering model
Attention is a key component of the now ubiquitous pre-trained language ...
research
∙
01/26/2021
Unsupervised clustering of series using dynamic programming and neural processes
Following the work of arXiv:2101.09512, we are interested in clustering ...
research
∙
01/23/2021
Unsupervised clustering of series using dynamic programming
We are interested in clustering parts of a given single multi-variate se...
research
∙
10/08/2020
On the importance of pre-training data volume for compact language models
Recent advances in language modeling have led to computationally intensi...
research
∙
07/09/2020
Fast Transformers with Clustered Attention
Transformers have been proven a successful model for a variety of tasks ...
research
∙
06/29/2020
Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Transformers achieve remarkable performance in several tasks but due to ...
research
∙
06/25/2020
Taming GANs with Lookahead
Generative Adversarial Networks are notoriously challenging to train. Th...
research
∙
06/16/2020
Gradient Alignment in Deep Neural Networks
One cornerstone of interpretable deep learning is the high degree of vis...
research
∙
04/06/2020
Fair Latency-Aware Metric for real-time video segmentation networks
As supervised semantic segmentation is reaching satisfying results, many...
research
∙
02/08/2020
Multi-task Reinforcement Learning with a Planning Quasi-Metric
We introduce a new reinforcement learning approach combining a planning ...
research
∙
10/25/2019
On the Tunability of Optimizers in Deep Learning
There is no consensus yet on the question whether adaptive gradient meth...
research
∙
05/03/2019
Processing Megapixel Images with Deep Attention-Sampling Models
Existing deep architectures cannot operate on very large signals such as...
research
∙
05/02/2019
Full-Jacobian Representation of Neural Networks
Non-linear functions such as neural networks can be locally approximated...
research
∙
04/18/2019
Reducing Noise in GAN Training with Variance Reduced Extragradient
Using large mini-batches when training generative adversarial networks (...
research
∙
06/05/2018
Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching
End-to-end deep-learning networks recently demonstrated extremely good p...
research
∙
03/02/2018
Not All Samples Are Created Equal: Deep Learning with Importance Sampling
Deep neural network training spends most of the computation on examples ...
research
∙
03/01/2018
Knowledge Transfer with Jacobian Matching
Classical distillation methods transfer representations from a "teacher"...
research
∙
02/12/2018
Geodesic Convolutional Shape Optimization
Aerodynamic shape optimization has many industrial applications. Existin...
research
∙
12/06/2017
SGAN: An Alternative Training of Generative Adversarial Networks
The Generative Adversarial Networks (GANs) have demonstrated impressive ...
research
∙
07/28/2017
The WILDTRACK Multi-Camera Person Dataset
People detection methods are highly sensitive to the perpetual occlusion...
research
∙
07/03/2017
Geometric calibration of Colour and Stereo Surface Imaging System of ESA's Trace Gas Orbiter
There are many geometric calibration methods for "standard" cameras. The...
research
∙
12/03/2016
Semi-supervised learning of deep metrics for stereo reconstruction
Deep-learning metrics have recently demonstrated extremely good performa...
research
∙
12/02/2016
Globally Consistent Multi-People Tracking using Motion Patterns
Many state-of-the-art approaches to people tracking rely on detecting th...
research
∙
11/28/2016
Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition
We present a unified framework for understanding human social behaviors ...
research
∙
11/23/2016
Multi-Modal Mean-Fields via Cardinality-Based Clamping
Mean Field inference is central to statistical physics. It has attracted...
research
∙
10/13/2016
Predicting the dynamics of 2d objects with a deep residual network
We investigate how a residual network can learn to predict the dynamics ...
research
∙
05/23/2016
A Sub-Quadratic Exact Medoid Algorithm
We present a new algorithm, trimed, for obtaining the medoid of a set, t...
research
∙
03/01/2016