b'Armand Joulin'

research

∙ 05/09/2023

ImageBind: One Embedding Space To Bind Them All

We present ImageBind, an approach to learn a joint embedding across six ...

0 Rohit Girdhar, et al. ∙

research

∙ 04/14/2023

DINOv2: Learning Robust Visual Features without Supervision

The recent breakthroughs in natural language processing for model pretra...

1 Maxime Oquab, et al. ∙

research

∙ 03/23/2023

The effectiveness of MAE pre-pretraining for billion-scale pretraining

This paper revisits the standard pretrain-then-finetune paradigm used in...

0 Mannat Singh, et al. ∙

research

∙ 02/27/2023

LLaMA: Open and Efficient Foundation Language Models

We introduce LLaMA, a collection of foundation language models ranging f...

6 Hugo Touvron, et al. ∙

research

∙ 08/05/2022

Few-shot Learning with Retrieval Augmented Language Models

Large language models have shown impressive few-shot results on a wide r...

10 Gautier Izacard, et al. ∙

research

∙ 07/08/2022

Improving Wikipedia Verifiability with AI

Verifiability is a core content policy of Wikipedia: claims that are lik...

12 Fabio Petroni, et al. ∙

research

∙ 06/16/2022

OmniMAE: Single Model Masked Pretraining on Images and Videos

Transformer-based architectures have become competitive across a variety...

11 Rohit Girdhar, et al. ∙

research

∙ 04/14/2022

Masked Siamese Networks for Label-Efficient Learning

We propose Masked Siamese Networks (MSN), a self-supervised learning fra...

25 Mahmoud Assran, et al. ∙

research

∙ 02/16/2022

Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision

Discriminative self-supervised learning allows training models on any ra...

70 Priya Goyal, et al. ∙

research

∙ 01/20/2022

Omnivore: A Single Model for Many Visual Modalities

Prior work has studied different visual modalities in isolation and deve...

7 Rohit Girdhar, et al. ∙

research

∙ 01/07/2022

Detecting Twenty-thousand Classes using Image-level Supervision

Current object detectors are limited in vocabulary size due to the small...

11 Xingyi Zhou, et al. ∙

research

∙ 12/27/2021

Augmenting Convolutional networks with attention-based aggregation

We show how to augment any convolutional network with an attention-based...

24 Hugo Touvron, et al. ∙

research

∙ 12/16/2021

Towards Unsupervised Dense Information Retrieval with Contrastive Learning

Information retrieval is an important component in natural language proc...

0 Gautier Izacard, et al. ∙

research

∙ 10/29/2021

Learning Co-segmentation by Segment Swapping for Retrieval and Discovery

The goal of this work is to efficiently identify visually similar patter...

3 Xi Shen, et al. ∙

research

∙ 09/16/2021

An End-to-End Transformer Model for 3D Object Detection

We propose 3DETR, an end-to-end Transformer based object detection model...

0 Ishan Misra, et al. ∙

research

∙ 06/17/2021

XCiT: Cross-Covariance Image Transformers

Following their success in natural language processing, transformers hav...

0 Alaaeldin El-Nouby, et al. ∙

research

∙ 05/07/2021

ResMLP: Feedforward networks for image classification with data-efficient training

We present ResMLP, an architecture built entirely upon multi-layer perce...

43 Hugo Touvron, et al. ∙

research

∙ 04/29/2021

Emerging Properties in Self-Supervised Vision Transformers

In this paper, we question if self-supervised learning provides new prop...

12 Mathilde Caron, et al. ∙

research

∙ 04/28/2021

Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples

This paper proposes a novel method of learning by predicting view assign...

0 Mahmoud Assran, et al. ∙

research

∙ 04/02/2021

LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference

We design a family of image classification architectures that optimize t...

0 Ben Graham, et al. ∙

research

∙ 03/02/2021

Self-supervised Pretraining of Visual Features in the Wild

Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and S...

0 Priya Goyal, et al. ∙

research

∙ 01/07/2021

Self-Supervised Pretraining of 3D Features on any Point-Cloud

Pretraining on large labeled datasets is a prerequisite to achieve good ...

14 Zaiwei Zhang, et al. ∙

research

∙ 10/21/2020

Beyond English-Centric Multilingual Machine Translation

Existing work in translation demonstrated the potential of massively mul...

11 Angela Fan, et al. ∙

research

∙ 09/21/2020

Target Conditioning for One-to-Many Generation

Neural Machine Translation (NMT) models often lack diversity in their ge...

19 Marie-Anne Lachaux, et al. ∙

research

∙ 06/17/2020

Unsupervised Learning of Visual Features by Contrasting Cluster Assignments

Unsupervised image representations have significantly reduced the gap wi...

0 Mathilde Caron, et al. ∙

research

∙ 04/15/2020

Training with Quantization Noise for Extreme Model Compression

We tackle the problem of producing compact models, maximizing their accu...

30 Angela Fan, et al. ∙

research

∙ 04/15/2020

Training with Quantization Noise for Extreme Fixed-Point Compression

We tackle the problem of producing compact models, maximizing their accu...

3 Angela Fan, et al. ∙

research

∙ 04/10/2020

Learning to Visually Navigate in Photorealistic Environments Without any Supervision

Learning to navigate in a realistic setting where an agent must rely sol...

3 Lina Mezghani, et al. ∙

research

∙ 02/21/2020

Accessing Higher-level Representations in Sequential Transformers with Feedback Memory

Transformers are feedforward networks that can process input tokens in p...

5 Angela Fan, et al. ∙

research

∙ 02/07/2020

Unsupervised pretraining transfers well across languages

Cross-lingual and multi-lingual training of Automatic Speech Recognition...

0 Morgane Riviere, et al. ∙

research

∙ 01/10/2020

Pruning Convolutional Neural Networks with Self-Supervision

Convolutional neural networks trained without supervision come close to ...

31 Mathilde Caron, et al. ∙

research

∙ 12/17/2019

Libri-Light: A Benchmark for ASR with Limited or No Supervision

We introduce a new collection of spoken English audio suitable for train...

0 Jacob Kahn, et al. ∙

research

∙ 11/10/2019

CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB

We show that margin-based bitext mining in a multilingual sentence space...

0 Holger Schwenk, et al. ∙

research

∙ 11/01/2019

CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data

Pre-training text representations have led to significant improvements i...

13 Guillaume Wenzek, et al. ∙

research

∙ 10/14/2019

Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment

In this paper, we focus on the problem of adapting word vector-based mod...

0 Piotr Bojanowski, et al. ∙

research

∙ 09/25/2019

Reducing Transformer Depth on Demand with Structured Dropout

Overparameterized transformer networks have obtained state of the art re...

10 Angela Fan, et al. ∙

research

∙ 07/22/2019

Why Build an Assistant in Minecraft?

In this document we describe a rationale for a research program aimed at...

2 Arthur Szlam, et al. ∙

research

∙ 07/12/2019

And the Bit Goes Down: Revisiting the Quantization of Neural Networks

In this paper, we address the problem of reducing the memory footprint o...

1 Pierre Stock, et al. ∙

research

∙ 07/02/2019

Augmenting Self-attention with Persistent Memory

Transformer networks have lead to important progress in language modelin...

6 Sainbayar Sukhbaatar, et al. ∙

research

∙ 05/19/2019

Adaptive Attention Span in Transformers

We propose a novel self-attention mechanism that can learn its optimal a...

0 Sainbayar Sukhbaatar, et al. ∙

research

∙ 05/03/2019

Leveraging Large-Scale Uncurated Data for Unsupervised Pre-training of Visual Features

Pre-training general-purpose visual features with convolutional neural n...

20 Mathilde Caron, et al. ∙

research

∙ 02/25/2019

Cooperative Learning of Disjoint Syntax and Semantics

There has been considerable attention devoted to models that learn to jo...

6 Serhii Havrylov, et al. ∙

research

∙ 11/02/2018

Unsupervised Hyperalignment for Multilingual Word Embeddings

We consider the problem of aligning continuous word representations, lea...

0 Jean Alaux, et al. ∙

research

∙ 07/15/2018

Deep Clustering for Unsupervised Learning of Visual Features

Clustering is a class of unsupervised learning methods that has been ext...

4 Mathilde Caron, et al. ∙

research

∙ 05/29/2018

Unsupervised Alignment of Embeddings with Wasserstein Procrustes

We consider the task of aligning two sets of points in high dimension, w...

2 Edouard Grave, et al. ∙

research

∙ 04/20/2018

Improving Supervised Bilingual Mapping of Word Embeddings

Continuous word representations, learned on different languages, can be ...

0 Armand Joulin, et al. ∙

research

∙ 02/19/2018

Learning Word Vectors for 157 Languages

Distributed word representations, or word vectors, have recently been ap...

0 Edouard Grave, et al. ∙

research

∙ 12/26/2017

Advances in Pre-Training Distributed Word Representations

Many Natural Language Processing applications nowadays rely on pre-train...

0 Tomas Mikolov, et al. ∙

research

∙ 11/07/2017

Unbounded cache model for online language modeling with open vocabulary

Recently, continuous cache models were proposed as extensions to recurre...

0 Edouard Grave, et al. ∙

research

∙ 10/30/2017

Fast Linear Model for Knowledge Graph Embeddings

This paper shows that a simple baseline based on a Bag-of-Words (BoW) re...

0 Armand Joulin, et al. ∙

Armand Joulin

Featured Co-authors