research
∙
05/09/2023
ImageBind: One Embedding Space To Bind Them All
We present ImageBind, an approach to learn a joint embedding across six ...
share
research
∙
04/14/2023
DINOv2: Learning Robust Visual Features without Supervision
The recent breakthroughs in natural language processing for model pretra...
research
∙
03/23/2023
The effectiveness of MAE pre-pretraining for billion-scale pretraining
This paper revisits the standard pretrain-then-finetune paradigm used in...
research
∙
02/27/2023
LLaMA: Open and Efficient Foundation Language Models
We introduce LLaMA, a collection of foundation language models ranging f...
research
∙
08/05/2022
Few-shot Learning with Retrieval Augmented Language Models
Large language models have shown impressive few-shot results on a wide r...
research
∙
07/08/2022
Improving Wikipedia Verifiability with AI
Verifiability is a core content policy of Wikipedia: claims that are lik...
research
∙
06/16/2022
OmniMAE: Single Model Masked Pretraining on Images and Videos
Transformer-based architectures have become competitive across a variety...
research
∙
04/14/2022
Masked Siamese Networks for Label-Efficient Learning
We propose Masked Siamese Networks (MSN), a self-supervised learning fra...
research
∙
02/16/2022
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision
Discriminative self-supervised learning allows training models on any ra...
research
∙
01/20/2022
Omnivore: A Single Model for Many Visual Modalities
Prior work has studied different visual modalities in isolation and deve...
research
∙
01/07/2022
Detecting Twenty-thousand Classes using Image-level Supervision
Current object detectors are limited in vocabulary size due to the small...
research
∙
12/27/2021
Augmenting Convolutional networks with attention-based aggregation
We show how to augment any convolutional network with an attention-based...
research
∙
12/16/2021
Towards Unsupervised Dense Information Retrieval with Contrastive Learning
Information retrieval is an important component in natural language proc...
research
∙
10/29/2021
Learning Co-segmentation by Segment Swapping for Retrieval and Discovery
The goal of this work is to efficiently identify visually similar patter...
research
∙
09/16/2021
An End-to-End Transformer Model for 3D Object Detection
We propose 3DETR, an end-to-end Transformer based object detection model...
research
∙
06/17/2021
XCiT: Cross-Covariance Image Transformers
Following their success in natural language processing, transformers hav...
research
∙
05/07/2021
ResMLP: Feedforward networks for image classification with data-efficient training
We present ResMLP, an architecture built entirely upon multi-layer perce...
research
∙
04/29/2021
Emerging Properties in Self-Supervised Vision Transformers
In this paper, we question if self-supervised learning provides new prop...
research
∙
04/28/2021
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
This paper proposes a novel method of learning by predicting view assign...
research
∙
04/02/2021
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
We design a family of image classification architectures that optimize t...
research
∙
03/02/2021
Self-supervised Pretraining of Visual Features in the Wild
Recently, self-supervised learning methods like MoCo, SimCLR, BYOL and S...
research
∙
01/07/2021
Self-Supervised Pretraining of 3D Features on any Point-Cloud
Pretraining on large labeled datasets is a prerequisite to achieve good ...
research
∙
10/21/2020
Beyond English-Centric Multilingual Machine Translation
Existing work in translation demonstrated the potential of massively mul...
research
∙
09/21/2020
Target Conditioning for One-to-Many Generation
Neural Machine Translation (NMT) models often lack diversity in their ge...
research
∙
06/17/2020
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Unsupervised image representations have significantly reduced the gap wi...
research
∙
04/15/2020
Training with Quantization Noise for Extreme Model Compression
We tackle the problem of producing compact models, maximizing their accu...
research
∙
04/15/2020
Training with Quantization Noise for Extreme Fixed-Point Compression
We tackle the problem of producing compact models, maximizing their accu...
research
∙
04/10/2020
Learning to Visually Navigate in Photorealistic Environments Without any Supervision
Learning to navigate in a realistic setting where an agent must rely sol...
research
∙
02/21/2020
Accessing Higher-level Representations in Sequential Transformers with Feedback Memory
Transformers are feedforward networks that can process input tokens in p...
research
∙
02/07/2020
Unsupervised pretraining transfers well across languages
Cross-lingual and multi-lingual training of Automatic Speech Recognition...
research
∙
01/10/2020
Pruning Convolutional Neural Networks with Self-Supervision
Convolutional neural networks trained without supervision come close to ...
research
∙
12/17/2019
Libri-Light: A Benchmark for ASR with Limited or No Supervision
We introduce a new collection of spoken English audio suitable for train...
research
∙
11/10/2019
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB
We show that margin-based bitext mining in a multilingual sentence space...
research
∙
11/01/2019
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Pre-training text representations have led to significant improvements i...
research
∙
10/14/2019
Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
In this paper, we focus on the problem of adapting word vector-based mod...
research
∙
09/25/2019
Reducing Transformer Depth on Demand with Structured Dropout
Overparameterized transformer networks have obtained state of the art re...
research
∙
07/22/2019
Why Build an Assistant in Minecraft?
In this document we describe a rationale for a research program aimed at...
research
∙
07/12/2019
And the Bit Goes Down: Revisiting the Quantization of Neural Networks
In this paper, we address the problem of reducing the memory footprint o...
research
∙
07/02/2019
Augmenting Self-attention with Persistent Memory
Transformer networks have lead to important progress in language modelin...
research
∙
05/19/2019
Adaptive Attention Span in Transformers
We propose a novel self-attention mechanism that can learn its optimal a...
research
∙
05/03/2019
Leveraging Large-Scale Uncurated Data for Unsupervised Pre-training of Visual Features
Pre-training general-purpose visual features with convolutional neural n...
research
∙
02/25/2019
Cooperative Learning of Disjoint Syntax and Semantics
There has been considerable attention devoted to models that learn to jo...
research
∙
11/02/2018
Unsupervised Hyperalignment for Multilingual Word Embeddings
We consider the problem of aligning continuous word representations, lea...
research
∙
07/15/2018
Deep Clustering for Unsupervised Learning of Visual Features
Clustering is a class of unsupervised learning methods that has been ext...
research
∙
05/29/2018
Unsupervised Alignment of Embeddings with Wasserstein Procrustes
We consider the task of aligning two sets of points in high dimension, w...
research
∙
04/20/2018
Improving Supervised Bilingual Mapping of Word Embeddings
Continuous word representations, learned on different languages, can be ...
research
∙
02/19/2018
Learning Word Vectors for 157 Languages
Distributed word representations, or word vectors, have recently been ap...
research
∙
12/26/2017
Advances in Pre-Training Distributed Word Representations
Many Natural Language Processing applications nowadays rely on pre-train...
research
∙
11/07/2017
Unbounded cache model for online language modeling with open vocabulary
Recently, continuous cache models were proposed as extensions to recurre...
research
∙
10/30/2017