research
∙
08/05/2023
Group Membership Bias
When learning to rank from user interactions, search and recommendation ...
share
research
∙
07/12/2023
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
The ubiquitous and demonstrably suboptimal choice of resizing images to ...
research
∙
04/24/2023
End-to-End Spatio-Temporal Action Localisation with Video Transformers
The most performant spatio-temporal action localisation models use exter...
research
∙
12/19/2022
DSI++: Updating Transformer Memory with New Documents
Differentiable Search Indices (DSIs) encode a corpus of documents in the...
research
∙
12/09/2022
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
Training large, deep neural networks to convergence can be prohibitively...
research
∙
10/20/2022
Transcending Scaling Laws with 0.1
Scaling language models improves performance but comes with significant ...
research
∙
10/14/2022
Λ-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells
Differentiable neural architecture search (DARTS) is a popular method fo...
research
∙
08/19/2022
Intersection of Parallels as an Early Stopping Criterion
A common way to avoid overfitting in supervised learning is early stoppi...
research
∙
07/21/2022
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
There have been a lot of interest in the scaling properties of Transform...
research
∙
07/08/2022
Beyond Transfer Learning: Co-finetuning for Action Localisation
Transfer learning is the predominant paradigm for training deep networks...
research
∙
05/02/2022
Retrieval-Enhanced Machine Learning
Although information access systems have long supported people in accomp...
research
∙
11/25/2021
PolyViT: Co-training Vision Transformers on Images, Videos and Audio
Can we train a single transformer model capable of processing multiple m...
research
∙
11/20/2021
Discrete Representations Strengthen Vision Transformer Robustness
Vision Transformer (ViT) is emerging as the state-of-the-art architectur...
research
∙
10/25/2021
The Efficiency Misnomer
Model efficiency is a critical aspect of developing and deploying machin...
research
∙
10/18/2021
SCENIC: A JAX Library for Computer Vision Research and Beyond
Scenic is an open-source JAX library with a focus on Transformer-based m...
research
∙
10/05/2021
Exploring the Limits of Large Scale Pre-training
Recent developments in large-scale machine learning suggest that by scal...
research
∙
09/22/2021
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
There remain many open questions pertaining to the scaling behaviour of ...
research
∙
07/14/2021
The Benchmark Lottery
The world of empirical machine learning (ML) strongly relies on benchmar...
research
∙
06/21/2021
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?
In this paper, we introduce a novel visual representation learning which...
research
∙
06/10/2021
Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent
We focus on the problem of domain adaptation when the goal is shifting t...
research
∙
06/08/2021
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks
State-of-the-art parameter-efficient fine-tuning methods rely on introdu...
research
∙
05/07/2021
Are Pre-trained Convolutions Better than Pre-trained Transformers?
In the era of pre-trained language models, Transformers are the de facto...
research
∙
03/29/2021
ViViT: A Video Vision Transformer
We present pure-transformer based models for video classification, drawi...
research
∙
03/01/2021
OmniNet: Omnidirectional Representations from Transformers
This paper proposes Omnidirectional Representations from Transformers (O...
research
∙
11/08/2020
Long Range Arena: A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely bec...
research
∙
10/22/2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
While the Transformer architecture has become the de-facto standard for ...
research
∙
09/14/2020
Efficient Transformers: A Survey
Transformer model architectures have garnered immense interest lately du...
research
∙
06/22/2020
IDF++: Analyzing and Improving Integer Discrete Flows for Lossless Compression
In this paper we analyse and improve integer discrete flows for lossless...
research
∙
05/31/2020
Transferring Inductive Biases through Knowledge Distillation
Having the right inductive biases can be crucial in many tasks or scenar...
research
∙
03/24/2020
MetNet: A Neural Weather Model for Precipitation Forecasting
Weather forecasting is a long standing scientific challenge with direct ...
research
∙
10/12/2018
HiTR: Hierarchical Topic Model Re-estimation for Measuring Topical Diversity of Documents
A high degree of topical diversity is often considered to be an importan...
research
∙
07/10/2018
Universal Transformers
Self-attentive feed-forward sequence models have been shown to achieve i...
research
∙
06/21/2018
Learning to Rank from Samples of Variable Quality
Training deep neural networks requires many training samples, but in pra...
research
∙
11/30/2017
Learning to Learn from Weak Supervision by Full Supervision
In this paper, we propose a method for training neural networks when we ...
research
∙
11/15/2017
Words are Malleable: Computing Semantic Shifts in Political and Media Discourse
Recently, researchers started to pay attention to the detection of tempo...
research
∙
11/08/2017
Fidelity-Weighted Learning
Training deep neural networks requires many training samples, but in pra...
research
∙
11/01/2017
Avoiding Your Teacher's Mistakes: Training Neural Networks with Controlled Weak Supervision
Training deep neural networks requires massive amounts of training data,...
research
∙
11/01/2017
On Search Powered Navigation
Query-based searching and browsing-based navigation are the two main com...
research
∙
08/11/2017
Learning to Attend, Copy, and Generate for Session-Based Query Suggestion
Users try to articulate their complex information needs during search se...
research
∙
07/24/2017
Share your Model instead of your Data: Privacy Preserving Mimic Learning for Ranking
Deep neural networks have become a primary tool for solving problems in ...
research
∙
07/13/2017
Neural Networks for Information Retrieval
Machine learning plays a role in many aspects of modern IR systems, and ...
research
∙
04/28/2017
Neural Ranking Models with Weak Supervision
Despite the impressive improvements achieved by unsupervised deep neural...
research
∙
09/02/2016