Armand Joulin

is this you? claim profile

∙ 0 followers
Facebook ∙ Stanford University

Research scientist at Facebook

XCiT: Cross-Covariance Image Transformers

Following their success in natural language processing, transformers hav...
06/17/2021 ∙ by Alaaeldin El-Nouby, et al. ∙ 0 ∙ share

read it
ResMLP: Feedforward networks for image classification with data-efficient training

We present ResMLP, an architecture built entirely upon multi-layer perce...
05/07/2021 ∙ by Hugo Touvron, et al. ∙ 43 ∙ share

read it
Emerging Properties in Self-Supervised Vision Transformers

In this paper, we question if self-supervised learning provides new prop...
04/29/2021 ∙ by Mathilde Caron, et al. ∙ 12 ∙ share

read it
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

This paper proposes a novel method of learning by predicting view assign...
04/28/2021 ∙ by Mahmoud Assran, et al. ∙ 0 ∙ share

read it
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference

We design a family of image classification architectures that optimize t...
04/02/2021 ∙ by Ben Graham, et al. ∙ 0 ∙ share

read it
Self-supervised Pretraining of Visual Features in the Wild

Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and S...
03/02/2021 ∙ by Priya Goyal, et al. ∙ 0 ∙ share

read it
Self-Supervised Pretraining of 3D Features on any Point-Cloud

Pretraining on large labeled datasets is a prerequisite to achieve good ...
01/07/2021 ∙ by Zaiwei Zhang, et al. ∙ 14 ∙ share

read it
Beyond English-Centric Multilingual Machine Translation

Existing work in translation demonstrated the potential of massively mul...
10/21/2020 ∙ by Angela Fan, et al. ∙ 11 ∙ share

read it
Target Conditioning for One-to-Many Generation

Neural Machine Translation (NMT) models often lack diversity in their ge...
09/21/2020 ∙ by Marie-Anne Lachaux, et al. ∙ 19 ∙ share

read it
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

Unsupervised image representations have significantly reduced the gap wi...
06/17/2020 ∙ by Mathilde Caron, et al. ∙ 0 ∙ share

read it
Training with Quantization Noise for Extreme Model Compression

We tackle the problem of producing compact models, maximizing their accu...
04/15/2020 ∙ by Angela Fan, et al. ∙ 30 ∙ share

read it
Training with Quantization Noise for Extreme Fixed-Point Compression

We tackle the problem of producing compact models, maximizing their accu...
04/15/2020 ∙ by Angela Fan, et al. ∙ 3 ∙ share

read it
Learning to Visually Navigate in Photorealistic Environments Without any Supervision

Learning to navigate in a realistic setting where an agent must rely sol...
04/10/2020 ∙ by Lina Mezghani, et al. ∙ 3 ∙ share

read it
Accessing Higher-level Representations in Sequential Transformers with Feedback Memory

Transformers are feedforward networks that can process input tokens in p...
02/21/2020 ∙ by Angela Fan, et al. ∙ 5 ∙ share

read it
Unsupervised pretraining transfers well across languages

Cross-lingual and multi-lingual training of Automatic Speech Recognition...
02/07/2020 ∙ by Morgane Riviere, et al. ∙ 0 ∙ share

read it
Pruning Convolutional Neural Networks with Self-Supervision

Convolutional neural networks trained without supervision come close to ...
01/10/2020 ∙ by Mathilde Caron, et al. ∙ 31 ∙ share

read it
Libri-Light: A Benchmark for ASR with Limited or No Supervision

We introduce a new collection of spoken English audio suitable for train...
12/17/2019 ∙ by Jacob Kahn, et al. ∙ 0 ∙ share

read it
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB

We show that margin-based bitext mining in a multilingual sentence space...
11/10/2019 ∙ by Holger Schwenk, et al. ∙ 0 ∙ share

read it
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data

Pre-training text representations have led to significant improvements i...
11/01/2019 ∙ by Guillaume Wenzek, et al. ∙ 13 ∙ share

read it
Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment

In this paper, we focus on the problem of adapting word vector-based mod...
10/14/2019 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it
Reducing Transformer Depth on Demand with Structured Dropout

Overparameterized transformer networks have obtained state of the art re...
09/25/2019 ∙ by Angela Fan, et al. ∙ 10 ∙ share

read it
Why Build an Assistant in Minecraft?

In this document we describe a rationale for a research program aimed at...
07/22/2019 ∙ by Arthur Szlam, et al. ∙ 2 ∙ share

read it
And the Bit Goes Down: Revisiting the Quantization of Neural Networks

In this paper, we address the problem of reducing the memory footprint o...
07/12/2019 ∙ by Pierre Stock, et al. ∙ 1 ∙ share

read it
Augmenting Self-attention with Persistent Memory

Transformer networks have lead to important progress in language modelin...
07/02/2019 ∙ by Sainbayar Sukhbaatar, et al. ∙ 6 ∙ share

read it
Adaptive Attention Span in Transformers

We propose a novel self-attention mechanism that can learn its optimal a...
05/19/2019 ∙ by Sainbayar Sukhbaatar, et al. ∙ 0 ∙ share

read it
Leveraging Large-Scale Uncurated Data for Unsupervised Pre-training of Visual Features

Pre-training general-purpose visual features with convolutional neural n...
05/03/2019 ∙ by Mathilde Caron, et al. ∙ 20 ∙ share

read it
Cooperative Learning of Disjoint Syntax and Semantics

There has been considerable attention devoted to models that learn to jo...
02/25/2019 ∙ by Serhii Havrylov, et al. ∙ 6 ∙ share

read it
Unsupervised Hyperalignment for Multilingual Word Embeddings

We consider the problem of aligning continuous word representations, lea...
11/02/2018 ∙ by Jean Alaux, et al. ∙ 0 ∙ share

read it
Deep Clustering for Unsupervised Learning of Visual Features

Clustering is a class of unsupervised learning methods that has been ext...
07/15/2018 ∙ by Mathilde Caron, et al. ∙ 4 ∙ share

read it
Unsupervised Alignment of Embeddings with Wasserstein Procrustes

We consider the task of aligning two sets of points in high dimension, w...
05/29/2018 ∙ by Edouard Grave, et al. ∙ 2 ∙ share

read it
Improving Supervised Bilingual Mapping of Word Embeddings

Continuous word representations, learned on different languages, can be ...
04/20/2018 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Learning Word Vectors for 157 Languages

Distributed word representations, or word vectors, have recently been ap...
02/19/2018 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Advances in Pre-Training Distributed Word Representations

Many Natural Language Processing applications nowadays rely on pre-train...
12/26/2017 ∙ by Tomas Mikolov, et al. ∙ 0 ∙ share

read it
Unbounded cache model for online language modeling with open vocabulary

Recently, continuous cache models were proposed as extensions to recurre...
11/07/2017 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Fast Linear Model for Knowledge Graph Embeddings

This paper shows that a simple baseline based on a Bag-of-Words (BoW) re...
10/30/2017 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Optimizing the Latent Space of Generative Networks

Generative Adversarial Networks (GANs) have been shown to be able to sam...
07/18/2017 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it
Unsupervised Learning by Predicting Noise

Convolutional neural networks provide visual features that perform remar...
04/18/2017 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it
CommAI: Evaluating the first steps towards a useful general AI

With machine learning successfully applied to new daunting problems almo...
01/31/2017 ∙ by Marco Baroni, et al. ∙ 0 ∙ share

read it
Learning Visual N-Grams from Web Data

Real-world image recognition systems need to recognize tens of thousands...
12/29/2016 ∙ by Ang Li, et al. ∙ 0 ∙ share

read it
Improving Neural Language Models with a Continuous Cache

We propose an extension to neural network language models to adapt their...
12/13/2016 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
FastText.zip: Compressing text classification models

We consider the problem of producing compact architectures for text clas...
12/12/2016 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Variable Computation in Recurrent Neural Networks

Recurrent neural networks (RNNs) have been used extensively and with inc...
11/18/2016 ∙ by Yacine Jernite, et al. ∙ 0 ∙ share

read it
Efficient softmax approximation for GPUs

We propose an approximate strategy to efficiently train neural network b...
09/14/2016 ∙ by Edouard Grave, et al. ∙ 0 ∙ share

read it
Enriching Word Vectors with Subword Information

Continuous word representations, trained on large unlabeled corpora are ...
07/15/2016 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it
Bag of Tricks for Efficient Text Classification

This paper explores a simple and efficient baseline for text classificat...
07/06/2016 ∙ by Armand Joulin, et al. ∙ 0 ∙ share

read it
Revisiting Visual Question Answering Baselines

Visual question answering (VQA) is an interesting learning setting for e...
06/27/2016 ∙ by Allan Jabri, et al. ∙ 0 ∙ share

read it
Locally-Optimized Inter-Subject Alignment of Functional Cortical Regions

Inter-subject registration of cortical areas is necessary in functional ...
06/07/2016 ∙ by Marius Cătălin Iordan, et al. ∙ 0 ∙ share

read it
A Roadmap towards Machine Intelligence

The development of intelligent machines is one of the biggest unsolved c...
11/25/2015 ∙ by Tomas Mikolov, et al. ∙ 0 ∙ share

read it
Learning Simple Algorithms from Examples

We present an approach for learning simple algorithms such as copying, m...
11/23/2015 ∙ by Wojciech Zaremba, et al. ∙ 0 ∙ share

read it
Alternative structures for character-level RNNs

Recurrent neural networks are convenient and efficient models for langua...
11/19/2015 ∙ by Piotr Bojanowski, et al. ∙ 0 ∙ share

read it

Armand Joulin

Featured Co-authors