research
∙
08/02/2023
From Sparse to Soft Mixtures of Experts
Sparse mixture of expert architectures (MoEs) scale model capacity witho...
share
research
∙
07/12/2023
Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
The ubiquitous and demonstrably suboptimal choice of resizing images to ...
research
∙
05/26/2023
Three Towers: Flexible Contrastive Learning with Pretrained Image Models
We introduce Three Towers (3T), a flexible method to improve the contras...
research
∙
03/27/2023
Sigmoid Loss for Language Image Pre-Training
We propose a simple pairwise sigmoid loss for image-text pre-training. U...
research
∙
01/30/2023
Massively Scaling Heteroscedastic Classifiers
Heteroscedastic classifiers, which learn a multivariate Gaussian distrib...
research
∙
12/15/2022
Image-and-Language Understanding from Pixels Only
Multimodal models are becoming increasingly effective, in part due to un...
research
∙
12/09/2022
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
Training large, deep neural networks to convergence can be prohibitively...
research
∙
09/14/2022
PaLI: A Jointly-Scaled Multilingual Language-Image Model
Effective scaling and a flexible task interface enable large language mo...
research
∙
06/06/2022
Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts
Large sparsely-activated models have obtained excellent performance in m...
research
∙
05/19/2022
Robust and Efficient Medical Imaging with Self-Supervision
Recent progress in Medical Artificial Intelligence (AI) has delivered sy...
research
∙
02/24/2022
Learning to Merge Tokens in Vision Transformers
Transformers are widely applied to solve natural language understanding ...
research
∙
11/15/2021
LiT: Zero-Shot Transfer with Locked-image Text Tuning
This paper presents contrastive-tuning, a simple method employing contra...
research
∙
10/07/2021
Sparse MoEs meet Efficient Ensembles
Machine learning models based on the aggregated outputs of submodels, ei...
research
∙
06/10/2021
Scaling Vision with Sparse Mixture of Experts
Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated exce...
research
∙
05/19/2021
Correlated Input-Dependent Label Noise in Large-Scale Image Classification
Large scale image classification datasets often contain noisy labels. We...
research
∙
04/08/2021
Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
We develop and rigorously evaluate a deep learning based system that can...
research
∙
01/14/2021
Supervised Transfer Learning at Scale for Medical Imaging
Transfer learning is a standard technique to improve performance on task...
research
∙
01/13/2021
Big Self-Supervised Models Advance Medical Image Classification
Self-supervised pretraining followed by supervised fine-tuning has seen ...
research
∙
10/14/2020
Deep Ensembles for Low-Data Transfer Learning
In the low-data regime, it is difficult to train good supervised models ...
research
∙
09/28/2020
Scalable Transfer Learning with Expert Models
Transfer of pre-trained representations can improve sample efficiency an...
research
∙
03/15/2020
Analysis of Softmax Approximation for Deep Classifiers under Input-Dependent Label Noise
Modelling uncertainty arising from input-dependent label noise is an inc...
research
∙
07/12/2018