b'Armand Joulin'

research

∙ 07/08/2022

Improving Wikipedia Verifiability with AI

Verifiability is a core content policy of Wikipedia: claims that are lik...

Fabio Petroni, et al. ∙

research

∙ 06/16/2022

OmniMAE: Single Model Masked Pretraining on Images and Videos

Transformer-based architectures have become competitive across a variety...

Rohit Girdhar, et al. ∙

research

∙ 04/14/2022

Masked Siamese Networks for Label-Efficient Learning

We propose Masked Siamese Networks (MSN), a self-supervised learning fra...

Mahmoud Assran, et al. ∙

research

∙ 02/16/2022

Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision

Discriminative self-supervised learning allows training models on any ra...

Priya Goyal, et al. ∙

research

∙ 01/20/2022

Omnivore: A Single Model for Many Visual Modalities

Prior work has studied different visual modalities in isolation and deve...

Rohit Girdhar, et al. ∙

research

∙ 01/07/2022

Detecting Twenty-thousand Classes using Image-level Supervision

Current object detectors are limited in vocabulary size due to the small...

Xingyi Zhou, et al. ∙

research

∙ 12/27/2021

Augmenting Convolutional networks with attention-based aggregation

We show how to augment any convolutional network with an attention-based...

Hugo Touvron, et al. ∙

research

∙ 12/16/2021

Towards Unsupervised Dense Information Retrieval with Contrastive Learning

Information retrieval is an important component in natural language proc...

Gautier Izacard, et al. ∙

research

∙ 10/29/2021

Learning Co-segmentation by Segment Swapping for Retrieval and Discovery

The goal of this work is to efficiently identify visually similar patter...

Xi Shen, et al. ∙

research

∙ 09/16/2021

An End-to-End Transformer Model for 3D Object Detection

We propose 3DETR, an end-to-end Transformer based object detection model...

Ishan Misra, et al. ∙

research

∙ 06/17/2021

XCiT: Cross-Covariance Image Transformers

Following their success in natural language processing, transformers hav...

Alaaeldin El-Nouby, et al. ∙

research

∙ 05/07/2021

ResMLP: Feedforward networks for image classification with data-efficient training

We present ResMLP, an architecture built entirely upon multi-layer perce...

Hugo Touvron, et al. ∙

research

∙ 04/29/2021

Emerging Properties in Self-Supervised Vision Transformers

In this paper, we question if self-supervised learning provides new prop...

Mathilde Caron, et al. ∙

research

∙ 04/28/2021

Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

This paper proposes a novel method of learning by predicting view assign...

Mahmoud Assran, et al. ∙

research

∙ 04/02/2021

LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference

We design a family of image classification architectures that optimize t...

Ben Graham, et al. ∙

research

∙ 03/02/2021

Self-supervised Pretraining of Visual Features in the Wild

Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and S...

Priya Goyal, et al. ∙

research

∙ 01/07/2021

Self-Supervised Pretraining of 3D Features on any Point-Cloud

Pretraining on large labeled datasets is a prerequisite to achieve good ...

Zaiwei Zhang, et al. ∙

research

∙ 10/21/2020

Beyond English-Centric Multilingual Machine Translation

Existing work in translation demonstrated the potential of massively mul...

Angela Fan, et al. ∙

research

∙ 09/21/2020

Target Conditioning for One-to-Many Generation

Neural Machine Translation (NMT) models often lack diversity in their ge...

Marie-Anne Lachaux, et al. ∙

research

∙ 06/17/2020

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

Unsupervised image representations have significantly reduced the gap wi...

Mathilde Caron, et al. ∙

research

∙ 04/15/2020

Training with Quantization Noise for Extreme Model Compression

We tackle the problem of producing compact models, maximizing their accu...

Angela Fan, et al. ∙

research

∙ 04/15/2020

Training with Quantization Noise for Extreme Fixed-Point Compression

We tackle the problem of producing compact models, maximizing their accu...

Angela Fan, et al. ∙

research

∙ 04/10/2020

Learning to Visually Navigate in Photorealistic Environments Without any Supervision

Learning to navigate in a realistic setting where an agent must rely sol...

Lina Mezghani, et al. ∙

research

∙ 02/21/2020

Accessing Higher-level Representations in Sequential Transformers with Feedback Memory

Transformers are feedforward networks that can process input tokens in p...

Angela Fan, et al. ∙

research

∙ 02/07/2020

Unsupervised pretraining transfers well across languages

Cross-lingual and multi-lingual training of Automatic Speech Recognition...

Morgane Riviere, et al. ∙

research

∙ 01/10/2020

Pruning Convolutional Neural Networks with Self-Supervision

Convolutional neural networks trained without supervision come close to ...

Mathilde Caron, et al. ∙

research

∙ 12/17/2019

Libri-Light: A Benchmark for ASR with Limited or No Supervision

We introduce a new collection of spoken English audio suitable for train...

Jacob Kahn, et al. ∙

research

∙ 11/10/2019

CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB

We show that margin-based bitext mining in a multilingual sentence space...

Holger Schwenk, et al. ∙

research

∙ 11/01/2019

CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data

Pre-training text representations have led to significant improvements i...

Guillaume Wenzek, et al. ∙

research

∙ 10/14/2019

Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment

In this paper, we focus on the problem of adapting word vector-based mod...

Piotr Bojanowski, et al. ∙

research

∙ 09/25/2019

Reducing Transformer Depth on Demand with Structured Dropout

Overparameterized transformer networks have obtained state of the art re...

Angela Fan, et al. ∙

research

∙ 07/22/2019

Why Build an Assistant in Minecraft?

In this document we describe a rationale for a research program aimed at...

Arthur Szlam, et al. ∙

research

∙ 07/12/2019

And the Bit Goes Down: Revisiting the Quantization of Neural Networks

In this paper, we address the problem of reducing the memory footprint o...

Pierre Stock, et al. ∙

research

∙ 07/02/2019

Augmenting Self-attention with Persistent Memory

Transformer networks have lead to important progress in language modelin...

Sainbayar Sukhbaatar, et al. ∙

research

∙ 05/19/2019

Adaptive Attention Span in Transformers

We propose a novel self-attention mechanism that can learn its optimal a...

Sainbayar Sukhbaatar, et al. ∙

research

∙ 05/03/2019

Leveraging Large-Scale Uncurated Data for Unsupervised Pre-training of Visual Features

Pre-training general-purpose visual features with convolutional neural n...

Mathilde Caron, et al. ∙

research

∙ 02/25/2019

Cooperative Learning of Disjoint Syntax and Semantics

There has been considerable attention devoted to models that learn to jo...

Serhii Havrylov, et al. ∙

research

∙ 11/02/2018

Unsupervised Hyperalignment for Multilingual Word Embeddings

We consider the problem of aligning continuous word representations, lea...

Jean Alaux, et al. ∙

research

∙ 07/15/2018

Deep Clustering for Unsupervised Learning of Visual Features

Clustering is a class of unsupervised learning methods that has been ext...

Mathilde Caron, et al. ∙

research

∙ 05/29/2018

Unsupervised Alignment of Embeddings with Wasserstein Procrustes

We consider the task of aligning two sets of points in high dimension, w...

Edouard Grave, et al. ∙

research

∙ 04/20/2018

Improving Supervised Bilingual Mapping of Word Embeddings

Continuous word representations, learned on different languages, can be ...

Armand Joulin, et al. ∙

research

∙ 02/19/2018

Learning Word Vectors for 157 Languages

Distributed word representations, or word vectors, have recently been ap...

Edouard Grave, et al. ∙

research

∙ 12/26/2017

Advances in Pre-Training Distributed Word Representations

Many Natural Language Processing applications nowadays rely on pre-train...

Tomas Mikolov, et al. ∙

research

∙ 11/07/2017

Unbounded cache model for online language modeling with open vocabulary

Recently, continuous cache models were proposed as extensions to recurre...

Edouard Grave, et al. ∙

research

∙ 10/30/2017

Fast Linear Model for Knowledge Graph Embeddings

This paper shows that a simple baseline based on a Bag-of-Words (BoW) re...

Armand Joulin, et al. ∙

research

∙ 07/18/2017

Optimizing the Latent Space of Generative Networks

Generative Adversarial Networks (GANs) have been shown to be able to sam...

Piotr Bojanowski, et al. ∙

research

∙ 04/18/2017

Unsupervised Learning by Predicting Noise

Convolutional neural networks provide visual features that perform remar...

Piotr Bojanowski, et al. ∙

research

∙ 01/31/2017

CommAI: Evaluating the first steps towards a useful general AI

With machine learning successfully applied to new daunting problems almo...

Marco Baroni, et al. ∙

research

∙ 12/29/2016

Learning Visual N-Grams from Web Data

Real-world image recognition systems need to recognize tens of thousands...

Ang Li, et al. ∙

research

∙ 12/13/2016

Improving Neural Language Models with a Continuous Cache

We propose an extension to neural network language models to adapt their...

Edouard Grave, et al. ∙

Armand Joulin

Featured Co-authors