b'Quoc V. Le'

research

∙ 03/23/2022

Revisiting Multi-Scale Feature Fusion for Semantic Segmentation

It is commonly believed that high internal resolution combined with expe...

Tianjian Meng, et al. ∙

research

∙ 02/21/2022

Transformer Quality in Linear Time

We revisit the design choices in Transformers, and propose methods to ad...

Weizhe Hua, et al. ∙

research

∙ 12/13/2021

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

Scaling language models with more data, compute and parameters has drive...

Nan Du, et al. ∙

research

∙ 11/19/2021

Combined Scaling for Zero-shot Transfer Learning

We present a combined scaling method called BASIC that achieves 85.7 zer...

Hieu Pham, et al. ∙

research

∙ 09/27/2021

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition

We summarize the results of a host of efforts using giant automatic spee...

Yu Zhang, et al. ∙

research

∙ 09/17/2021

Primer: Searching for Efficient Transformers for Language Modeling

Large Transformer models have been central to recent advances in natural...

David R. So, et al. ∙

research

∙ 09/13/2021

STraTA: Self-Training with Task Augmentation for Better Few-shot Learning

Despite their recent successes in tackling many NLP tasks, large-scale p...

Tu Vu, et al. ∙

research

∙ 09/03/2021

Finetuned Language Models Are Zero-Shot Learners

This paper explores a simple method for improving the zero-shot learning...

Jason Wei, et al. ∙

research

∙ 08/25/2021

Multi-Task Self-Training for Learning General Representations

Despite the fast progress in training specialized models for various tas...

Golnaz Ghiasi, et al. ∙

research

∙ 06/09/2021

CoAtNet: Marrying Convolution and Attention for All Data Sizes

Transformers have attracted increasing interests in computer vision, but...

Zihang Dai, et al. ∙

research

∙ 05/17/2021

Pay Attention to MLPs

Transformers have become one of the most important architectural innovat...

Hanxiao Liu, et al. ∙

research

∙ 04/01/2021

EfficientNetV2: Smaller Models and Faster Training

This paper introduces EfficientNetV2, a new family of convolutional netw...

Mingxing Tan, et al. ∙

research

∙ 02/11/2021

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Pre-trained representations are becoming crucial for many NLP and percep...

Chao Jia, et al. ∙

research

∙ 01/21/2021

PyGlove: Symbolic Programming for Automated Machine Learning

Neural networks are sensitive to hyper-parameter and architecture choice...

Daiyi Peng, et al. ∙

research

∙ 01/08/2021

Evolving Reinforcement Learning Algorithms

We propose a method for meta-learning reinforcement learning algorithms ...

John D. Co-Reyes, et al. ∙

research

∙ 01/05/2021

AutoDropout: Learning Dropout Patterns to Regularize Deep Networks

Neural networks are often over-parameterized and hence benefit from aggr...

Hieu Pham, et al. ∙

research

∙ 12/15/2020

Pre-Training Transformers as Energy-Based Cloze Models

We introduce Electric, an energy-based cloze model for representation le...

Kevin Clark, et al. ∙

research

∙ 12/13/2020

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

Building instance segmentation models that are data-efficient and can ha...

Golnaz Ghiasi, et al. ∙

research

∙ 11/09/2020

Towards Domain-Agnostic Contrastive Learning

Despite recent success, most contrastive self-supervised learning method...

Vikas Verma, et al. ∙

research

∙ 10/20/2020

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

We employ a combination of recent developments in semi-supervised learni...

Yu Zhang, et al. ∙

research

∙ 06/25/2020

Smooth Adversarial Training

It is commonly believed that networks cannot be both accurate and robust...

Cihang Xie, et al. ∙

research

∙ 06/11/2020

Rethinking Pre-training and Self-training

Pre-training is a dominant paradigm in computer vision. For example, sup...

Barret Zoph, et al. ∙

research

∙ 06/05/2020

AutoHAS: Differentiable Hyper-parameter and Architecture Search

Neural Architecture Search (NAS) has achieved significant progress in pu...

Xuanyi Dong, et al. ∙

research

∙ 06/05/2020

Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing

With the success of language pretraining, it is highly desirable to deve...

Zihang Dai, et al. ∙

research

∙ 05/19/2020

Improved Noisy Student Training for Automatic Speech Recognition

Recently, a semi-supervised learning method known as "noisy student trai...

Daniel S. Park, et al. ∙

research

∙ 04/22/2020

Chip Placement with Deep Reinforcement Learning

In this work, we present a learning-based approach to chip placement, on...

Azalia Mirhoseini, et al. ∙

research

∙ 04/06/2020

Evolving Normalization-Activation Layers

Normalization layers and activation functions are critical components in...

Hanxiao Liu, et al. ∙

research

∙ 04/02/2020

Improving 3D Object Detection through Progressive Population Based Augmentation

Data augmentation has been widely adopted for object detection in 3D poi...

Shuyang Cheng, et al. ∙

research

∙ 03/23/2020

Meta Pseudo Labels

Many training algorithms of a deep neural network can be interpreted as ...

Hieu Pham, et al. ∙

research

∙ 03/23/2020

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Masked language modeling (MLM) pre-training methods such as BERT corrupt...

Kevin Clark, et al. ∙

research

∙ 03/06/2020

AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

Machine learning research has advanced in multiple aspects, including mo...

Esteban Real, et al. ∙

research

∙ 01/27/2020

Towards a Human-like Open-Domain Chatbot

We present Meena, a multi-turn open-domain chatbot trained end-to-end on...

Daniel Adiwardana, et al. ∙

research

∙ 12/11/2019

SpecAugment on Large Scale Datasets

Recently, SpecAugment, an augmentation scheme for automatic speech recog...

Daniel S. Park, et al. ∙

research

∙ 12/10/2019

SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

Convolutional neural networks typically encode an input image into a ser...

Xianzhi Du, et al. ∙

research

∙ 12/02/2019

MnasFPN: Learning Latency-aware Pyramid Architecture for Object Detection on Mobile Devices

Despite the blooming success of architecture search for vision tasks in ...

Bo Chen, et al. ∙

research

∙ 11/21/2019

Adversarial Examples Improve Image Recognition

Adversarial examples are commonly viewed as a threat to ConvNets. Here w...

Cihang Xie, et al. ∙

research

∙ 11/20/2019

EfficientDet: Scalable and Efficient Object Detection

Model efficiency has become increasingly important in computer vision. I...

Mingxing Tan, et al. ∙

research

∙ 11/11/2019

Self-training with Noisy Student improves ImageNet classification

We present a simple self-training method that achieves 87.4 on ImageNet,...

Qizhe Xie, et al. ∙

research

∙ 11/05/2019

High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks

Predicting future video frames is extremely challenging, as there are ma...

Ruben Villegas, et al. ∙

research

∙ 09/30/2019

RandAugment: Practical data augmentation with no separate search

Recent work has shown that data augmentation has the potential to signif...

Ekin D. Cubuk, et al. ∙

research

∙ 08/20/2019

Saccader: Improving Accuracy of Hard Attention Models for Vision

Although deep convolutional neural networks achieve state-of-the-art per...

Gamaleldin F. Elsayed, et al. ∙

research

∙ 07/22/2019

MixNet: Mixed Depthwise Convolutional Kernels

Depthwise convolution is becoming increasingly popular in modern efficie...

Mingxing Tan, et al. ∙

research

∙ 07/10/2019

BAM! Born-Again Multi-Task Networks for Natural Language Understanding

It can be challenging to train multi-task neural networks that outperfor...

Kevin Clark, et al. ∙

research

∙ 07/10/2019

Neural Input Search for Large Scale Recommendation Models

Recommendation problems with large numbers of discrete items, such as pr...

Manas R. Joglekar, et al. ∙

research

∙ 06/26/2019

Learning Data Augmentation Strategies for Object Detection

Data augmentation is a critical component of training deep learning mode...

Barret Zoph, et al. ∙

research

∙ 06/19/2019

XLNet: Generalized Autoregressive Pretraining for Language Understanding

With the capability of modeling bidirectional contexts, denoising autoen...

Zhilin Yang, et al. ∙

research

∙ 06/07/2019

Selfie: Self-supervised Pretraining for Image Embedding

We introduce a pretraining technique called Selfie, which stands for SEL...

Trieu H. Trinh, et al. ∙

research

∙ 05/28/2019

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Convolutional Neural Networks (ConvNets) are commonly developed at a fix...

Mingxing Tan, et al. ∙

research

∙ 05/09/2019

The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study

We investigate how the final parameters found by stochastic gradient des...

Daniel S. Park, et al. ∙

research

∙ 05/06/2019

Searching for MobileNetV3

We present the next generation of MobileNets based on a combination of c...

Andrew Howard, et al. ∙

Quoc V. Le

Featured Co-authors