Edouard Grave

is this you? claim profile

∙ 0 followers
Facebook ∙ Inria

Research Scientist at Facebook AI Research

Training with Quantization Noise for Extreme Model Compression

We tackle the problem of producing compact models, maximizing their accu...
04/15/2020 ∙ by Angela Fan, et al. ∙ 30 ∙ share

read it
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data

Pre-training text representations have led to significant improvements i...
11/01/2019 ∙ by Guillaume Wenzek, et al. ∙ 13 ∙ share

read it
Reducing Transformer Depth on Demand with Structured Dropout

Overparameterized transformer networks have obtained state of the art re...
09/25/2019 ∙ by Angela Fan, et al. ∙ 10 ∙ share

read it
Augmenting Self-attention with Persistent Memory

Transformer networks have lead to important progress in language modelin...
07/02/2019 ∙ by Sainbayar Sukhbaatar, et al. ∙ 6 ∙ share

read it
Accessing Higher-level Representations in Sequential Transformers with Feedback Memory

Transformers are feedforward networks that can process input tokens in p...
02/21/2020 ∙ by Angela Fan, et al. ∙ 5 ∙ share

read it
Training with Quantization Noise for Extreme Fixed-Point Compression

We tackle the problem of producing compact models, maximizing their accu...
04/15/2020 ∙ by Angela Fan, et al. ∙ 3 ∙ share

read it
Unsupervised Alignment of Embeddings with Wasserstein Procrustes

We consider the task of aligning two sets of points in high dimension, w...
05/29/2018 ∙ by Edouard Grave, et al. ∙ 2 ∙ share

read it
Fast Linear Model for Knowledge Graph Embeddings

This paper shows that a simple baseline based on a Bag-of-Words (BoW) re...
10/30/2017 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Parseval Networks: Improving Robustness to Adversarial Examples

We introduce Parseval networks, a form of deep neural networks in which ...
04/28/2017 ∙ by Moustapha Cisse, et al. ∙ 0 ∙ share

read it
Trace Lasso: a trace norm regularization for correlated designs

Using the ℓ_1-norm to regularize the estimation of the parameter vector ...
09/09/2011 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Unbounded cache model for online language modeling with open vocabulary

Recently, continuous cache models were proposed as extensions to recurre...
11/07/2017 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Improving Neural Language Models with a Continuous Cache

We propose an extension to neural network language models to adapt their...
12/13/2016 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
FastText.zip: Compressing text classification models

We consider the problem of producing compact architectures for text clas...
12/12/2016 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Variable Computation in Recurrent Neural Networks

Recurrent neural networks (RNNs) have been used extensively and with inc...
11/18/2016 ∙ by Yacine Jernite, et al. ∙ 0 ∙ share

read it
Efficient softmax approximation for GPUs

We propose an approximate strategy to efficiently train neural network b...
09/14/2016 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Enriching Word Vectors with Subword Information

Continuous word representations, trained on large unlabeled corpora are ...
07/15/2016 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it
Bag of Tricks for Efficient Text Classification

This paper explores a simple and efficient baseline for text classificat...
07/06/2016 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Longitudinal Analysis of Discussion Topics in an Online Breast Cancer Community using Convolutional Neural Networks

Identifying topics of discussions in online health communities (OHC) is ...
03/28/2016 ∙ by Shaodian Zhang, et al. ∙ 0 ∙ share

read it
Advances in Pre-Training Distributed Word Representations

Many Natural Language Processing applications nowadays rely on pre-train...
12/26/2017 ∙ by Tomas Mikolov, et al. ∙ 0 ∙ share

read it
Learning Word Vectors for 157 Languages

Distributed word representations, or word vectors, have recently been ap...
02/19/2018 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Colorless green recurrent networks dream hierarchically

Recurrent neural networks (RNNs) have achieved impressive results in a v...
03/29/2018 ∙ by Kristina Gulordava, et al. ∙ 0 ∙ share

read it
Improving Supervised Bilingual Mapping of Word Embeddings

Continuous word representations, learned on different languages, can be ...
04/20/2018 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Lightweight Adaptive Mixture of Neural and N-gram Language Models

It is often the case that the best performing language model is an ensem...
04/20/2018 ∙ by Anton Bakhtin, et al. ∙ 0 ∙ share

read it
Unsupervised Hyperalignment for Multilingual Word Embeddings

We consider the problem of aligning continuous word representations, lea...
11/02/2018 ∙ by Jean Alaux, et al. ∙ 0 ∙ share

read it
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling

Work on the problem of contextualized word representation -- the develop...
12/28/2018 ∙ by Samuel R. Bowman, et al. ∙ 0 ∙ share

read it
Adaptive Attention Span in Transformers

We propose a novel self-attention mechanism that can learn its optimal a...
05/19/2019 ∙ by Sainbayar Sukhbaatar, et al. ∙ 0 ∙ share

read it
Misspelling Oblivious Word Embeddings

In this paper we present a method to learn word embeddings that are resi...
05/23/2019 ∙ by Bora Edizel, et al. ∙ 0 ∙ share

read it
Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction

Human translators routinely have to translate rare inflections of words ...
09/06/2019 ∙ by Paula Czarnowska, et al. ∙ 0 ∙ share

read it
Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment

In this paper, we focus on the problem of adapting word vector-based mod...
10/14/2019 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it
Depth-Adaptive Transformer

State of the art sequence-to-sequence models perform a fixed number of c...
10/22/2019 ∙ by Maha Elbayad, et al. ∙ 0 ∙ share

read it
Unsupervised Cross-lingual Representation Learning at Scale

This paper shows that pretraining multilingual language models at scale ...
11/05/2019 ∙ by Alexis Conneau, et al. ∙ 0 ∙ share

read it
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB

We show that margin-based bitext mining in a multilingual sentence space...
11/10/2019 ∙ by Holger Schwenk, et al. ∙ 0 ∙ share

read it
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures

We study ResNet-, Time-Depth Separable ConvNets-, and Transformer-based ...
11/19/2019 ∙ by Gabriel Synnaeve, et al. ∙ 0 ∙ share

read it

Edouard Grave

Featured Co-authors