Basil Mustafa | DeepAI

DeepAI AI Chat

Featured Co-authors

Xi Chen
293 publications
Xiao Wang
144 publications
Yun Liu
88 publications
Yi Tay
76 publications
Yuan Liu
73 publications
Mohammad Norouzi
71 publications
Jie Ren
66 publications
Ting Chen
65 publications
Ender Konukoglu
57 publications
Anelia Angelova
53 publications
Balaji Lakshminarayanan
51 publications

∙ 08/02/2023

From Sparse to Soft Mixtures of Experts

Sparse mixture of expert architectures (MoEs) scale model capacity witho...

0 Joan Puigcerver, et al. ∙

∙ 07/12/2023

Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution

The ubiquitous and demonstrably suboptimal choice of resizing images to ...

0 Mostafa Dehghani, et al. ∙

∙ 05/26/2023

Three Towers: Flexible Contrastive Learning with Pretrained Image Models

We introduce Three Towers (3T), a flexible method to improve the contras...

0 Jannik Kossen, et al. ∙

∙ 03/27/2023

Sigmoid Loss for Language Image Pre-Training

We propose a simple pairwise sigmoid loss for image-text pre-training. U...

0 Xiaohua Zhai, et al. ∙

∙ 01/30/2023

Massively Scaling Heteroscedastic Classifiers

Heteroscedastic classifiers, which learn a multivariate Gaussian distrib...

0 Mark Collier, et al. ∙

∙ 12/15/2022

Image-and-Language Understanding from Pixels Only

Multimodal models are becoming increasingly effective, in part due to un...

0 Michael Tschannen, et al. ∙

∙ 12/09/2022

Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints

Training large, deep neural networks to convergence can be prohibitively...

0 Aran Komatsuzaki, et al. ∙

∙ 09/14/2022

PaLI: A Jointly-Scaled Multilingual Language-Image Model

Effective scaling and a flexible task interface enable large language mo...

6 Xi Chen, et al. ∙

∙ 06/06/2022

Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts

Large sparsely-activated models have obtained excellent performance in m...

6 Basil Mustafa, et al. ∙

∙ 05/19/2022

Robust and Efficient Medical Imaging with Self-Supervision

Recent progress in Medical Artificial Intelligence (AI) has delivered sy...

24 Shekoofeh Azizi, et al. ∙

∙ 02/24/2022

Learning to Merge Tokens in Vision Transformers

Transformers are widely applied to solve natural language understanding ...

3 Cedric Renggli, et al. ∙

∙ 11/15/2021

LiT: Zero-Shot Transfer with Locked-image Text Tuning

This paper presents contrastive-tuning, a simple method employing contra...

0 Xiaohua Zhai, et al. ∙

∙ 10/07/2021

Sparse MoEs meet Efficient Ensembles

Machine learning models based on the aggregated outputs of submodels, ei...

3 James Urquhart Allingham, et al. ∙

∙ 06/10/2021

Scaling Vision with Sparse Mixture of Experts

Sparsely-gated Mixture of Experts networks (MoEs) have demonstrated exce...

6 Carlos Riquelme, et al. ∙

∙ 05/19/2021

Correlated Input-Dependent Label Noise in Large-Scale Image Classification

Large scale image classification datasets often contain noisy labels. We...

0 Mark Collier, et al. ∙

∙ 04/08/2021

Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions

We develop and rigorously evaluate a deep learning based system that can...

27 Abhijit Guha Roy, et al. ∙

∙ 01/14/2021

Supervised Transfer Learning at Scale for Medical Imaging

Transfer learning is a standard technique to improve performance on task...

30 Basil Mustafa, et al. ∙

∙ 01/13/2021

Big Self-Supervised Models Advance Medical Image Classification

Self-supervised pretraining followed by supervised fine-tuning has seen ...

22 Shekoofeh Azizi, et al. ∙

∙ 10/14/2020

Deep Ensembles for Low-Data Transfer Learning

In the low-data regime, it is difficult to train good supervised models ...

19 Basil Mustafa, et al. ∙

∙ 09/28/2020

Scalable Transfer Learning with Expert Models

Transfer of pre-trained representations can improve sample efficiency an...

11 Joan Puigcerver, et al. ∙

∙ 03/15/2020

Analysis of Softmax Approximation for Deep Classifiers under Input-Dependent Label Noise

Modelling uncertainty arising from input-dependent label noise is an inc...

5 Mark Collier, et al. ∙

∙ 07/12/2018

Learning to Segment Medical Images with Scribble-Supervision Alone

Semantic segmentation of medical images is a crucial step for the quanti...

6 Yigit B. Can, et al. ∙

Are you a researcher?
Expose your work
to one of the largest
A.I. communities in the world