Quoc V. Le

is this you? claim profile

∙ 0 followers
Google

Research Scientist at Google.

CoAtNet: Marrying Convolution and Attention for All Data Sizes

Transformers have attracted increasing interests in computer vision, but...
06/09/2021 ∙ by Zihang Dai, et al. ∙ 0 ∙ share

read it
Pay Attention to MLPs

Transformers have become one of the most important architectural innovat...
05/17/2021 ∙ by Hanxiao Liu, et al. ∙ 39 ∙ share

read it
EfficientNetV2: Smaller Models and Faster Training

This paper introduces EfficientNetV2, a new family of convolutional netw...
04/01/2021 ∙ by Mingxing Tan, et al. ∙ 0 ∙ share

read it
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

Pre-trained representations are becoming crucial for many NLP and percep...
02/11/2021 ∙ by Chao Jia, et al. ∙ 0 ∙ share

read it
PyGlove: Symbolic Programming for Automated Machine Learning

Neural networks are sensitive to hyper-parameter and architecture choice...
01/21/2021 ∙ by Daiyi Peng, et al. ∙ 30 ∙ share

read it
Evolving Reinforcement Learning Algorithms

We propose a method for meta-learning reinforcement learning algorithms ...
01/08/2021 ∙ by John D. Co-Reyes, et al. ∙ 28 ∙ share

read it
AutoDropout: Learning Dropout Patterns to Regularize Deep Networks

Neural networks are often over-parameterized and hence benefit from aggr...
01/05/2021 ∙ by Hieu Pham, et al. ∙ 7 ∙ share

read it
Pre-Training Transformers as Energy-Based Cloze Models

We introduce Electric, an energy-based cloze model for representation le...
12/15/2020 ∙ by Kevin Clark, et al. ∙ 0 ∙ share

read it
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

Building instance segmentation models that are data-efficient and can ha...
12/13/2020 ∙ by Golnaz Ghiasi, et al. ∙ 18 ∙ share

read it
Towards Domain-Agnostic Contrastive Learning

Despite recent success, most contrastive self-supervised learning method...
11/09/2020 ∙ by Vikas Verma, et al. ∙ 0 ∙ share

read it
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

We employ a combination of recent developments in semi-supervised learni...
10/20/2020 ∙ by Yu Zhang, et al. ∙ 0 ∙ share

read it
Smooth Adversarial Training

It is commonly believed that networks cannot be both accurate and robust...
06/25/2020 ∙ by Cihang Xie, et al. ∙ 13 ∙ share

read it
Rethinking Pre-training and Self-training

Pre-training is a dominant paradigm in computer vision. For example, sup...
06/11/2020 ∙ by Barret Zoph, et al. ∙ 0 ∙ share

read it
AutoHAS: Differentiable Hyper-parameter and Architecture Search

Neural Architecture Search (NAS) has achieved significant progress in pu...
06/05/2020 ∙ by Xuanyi Dong, et al. ∙ 0 ∙ share

read it
Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processing

With the success of language pretraining, it is highly desirable to deve...
06/05/2020 ∙ by Zihang Dai, et al. ∙ 0 ∙ share

read it
Improved Noisy Student Training for Automatic Speech Recognition

Recently, a semi-supervised learning method known as "noisy student trai...
05/19/2020 ∙ by Daniel S. Park, et al. ∙ 0 ∙ share

read it
Chip Placement with Deep Reinforcement Learning

In this work, we present a learning-based approach to chip placement, on...
04/22/2020 ∙ by Azalia Mirhoseini, et al. ∙ 9 ∙ share

read it
Evolving Normalization-Activation Layers

Normalization layers and activation functions are critical components in...
04/06/2020 ∙ by Hanxiao Liu, et al. ∙ 9 ∙ share

read it
Improving 3D Object Detection through Progressive Population Based Augmentation

Data augmentation has been widely adopted for object detection in 3D poi...
04/02/2020 ∙ by Shuyang Cheng, et al. ∙ 1 ∙ share

read it
Meta Pseudo Labels

Many training algorithms of a deep neural network can be interpreted as ...
03/23/2020 ∙ by Hieu Pham, et al. ∙ 0 ∙ share

read it
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

Masked language modeling (MLM) pre-training methods such as BERT corrupt...
03/23/2020 ∙ by Kevin Clark, et al. ∙ 0 ∙ share

read it
AutoML-Zero: Evolving Machine Learning Algorithms From Scratch

Machine learning research has advanced in multiple aspects, including mo...
03/06/2020 ∙ by Esteban Real, et al. ∙ 32 ∙ share

read it
Towards a Human-like Open-Domain Chatbot

We present Meena, a multi-turn open-domain chatbot trained end-to-end on...
01/27/2020 ∙ by Daniel Adiwardana, et al. ∙ 16 ∙ share

read it
SpecAugment on Large Scale Datasets

Recently, SpecAugment, an augmentation scheme for automatic speech recog...
12/11/2019 ∙ by Daniel S. Park, et al. ∙ 0 ∙ share

read it
SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

Convolutional neural networks typically encode an input image into a ser...
12/10/2019 ∙ by Xianzhi Du, et al. ∙ 0 ∙ share

read it
MnasFPN: Learning Latency-aware Pyramid Architecture for Object Detection on Mobile Devices

Despite the blooming success of architecture search for vision tasks in ...
12/02/2019 ∙ by Bo Chen, et al. ∙ 0 ∙ share

read it
Adversarial Examples Improve Image Recognition

Adversarial examples are commonly viewed as a threat to ConvNets. Here w...
11/21/2019 ∙ by Cihang Xie, et al. ∙ 24 ∙ share

read it
EfficientDet: Scalable and Efficient Object Detection

Model efficiency has become increasingly important in computer vision. I...
11/20/2019 ∙ by Mingxing Tan, et al. ∙ 0 ∙ share

read it
Self-training with Noisy Student improves ImageNet classification

We present a simple self-training method that achieves 87.4 on ImageNet,...
11/11/2019 ∙ by Qizhe Xie, et al. ∙ 36 ∙ share

read it
High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks

Predicting future video frames is extremely challenging, as there are ma...
11/05/2019 ∙ by Ruben Villegas, et al. ∙ 37 ∙ share

read it
RandAugment: Practical data augmentation with no separate search

Recent work has shown that data augmentation has the potential to signif...
09/30/2019 ∙ by Ekin D. Cubuk, et al. ∙ 26 ∙ share

read it
Saccader: Improving Accuracy of Hard Attention Models for Vision

Although deep convolutional neural networks achieve state-of-the-art per...
08/20/2019 ∙ by Gamaleldin F. Elsayed, et al. ∙ 9 ∙ share

read it
MixNet: Mixed Depthwise Convolutional Kernels

Depthwise convolution is becoming increasingly popular in modern efficie...
07/22/2019 ∙ by Mingxing Tan, et al. ∙ 0 ∙ share

read it
BAM! Born-Again Multi-Task Networks for Natural Language Understanding

It can be challenging to train multi-task neural networks that outperfor...
07/10/2019 ∙ by Kevin Clark, et al. ∙ 0 ∙ share

read it
Neural Input Search for Large Scale Recommendation Models

Recommendation problems with large numbers of discrete items, such as pr...
07/10/2019 ∙ by Manas R. Joglekar, et al. ∙ 0 ∙ share

read it
Learning Data Augmentation Strategies for Object Detection

Data augmentation is a critical component of training deep learning mode...
06/26/2019 ∙ by Barret Zoph, et al. ∙ 4 ∙ share

read it
XLNet: Generalized Autoregressive Pretraining for Language Understanding

With the capability of modeling bidirectional contexts, denoising autoen...
06/19/2019 ∙ by Zhilin Yang, et al. ∙ 0 ∙ share

read it
Selfie: Self-supervised Pretraining for Image Embedding

We introduce a pretraining technique called Selfie, which stands for SEL...
06/07/2019 ∙ by Trieu H. Trinh, et al. ∙ 2 ∙ share

read it
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Convolutional Neural Networks (ConvNets) are commonly developed at a fix...
05/28/2019 ∙ by Mingxing Tan, et al. ∙ 1 ∙ share

read it
The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study

We investigate how the final parameters found by stochastic gradient des...
05/09/2019 ∙ by Daniel S. Park, et al. ∙ 0 ∙ share

read it
Searching for MobileNetV3

We present the next generation of MobileNets based on a combination of c...
05/06/2019 ∙ by Andrew Howard, et al. ∙ 53 ∙ share

read it
Unsupervised Data Augmentation

Despite its success, deep learning still needs large labeled datasets to...
04/29/2019 ∙ by Qizhe Xie, et al. ∙ 18 ∙ share

read it
Attention Augmented Convolutional Networks

Convolutional networks have been the paradigm of choice in many computer...
04/22/2019 ∙ by Irwan Bello, et al. ∙ 30 ∙ share

read it
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

We present SpecAugment, a simple data augmentation method for speech rec...
04/18/2019 ∙ by Daniel S. Park, et al. ∙ 16 ∙ share

read it
NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection

Current state-of-the-art convolutional architectures for object detectio...
04/16/2019 ∙ by Golnaz Ghiasi, et al. ∙ 0 ∙ share

read it
Soft Conditional Computation

Conditional computation aims to increase the size and accuracy of a netw...
04/10/2019 ∙ by Brandon Yang, et al. ∙ 0 ∙ share

read it
The Evolved Transformer

Recent works have highlighted the strengths of the Transformer architect...
01/30/2019 ∙ by David R. So, et al. ∙ 0 ∙ share

read it
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

Transformer networks have a potential of learning longer-term dependency...
01/09/2019 ∙ by Zihang Dai, et al. ∙ 2 ∙ share

read it
Domain Adaptive Transfer Learning with Specialist Models

Transfer learning is a widely used method to build high performing compu...
11/16/2018 ∙ by Jiquan Ngiam, et al. ∙ 0 ∙ share

read it
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

GPipe is a scalable pipeline parallelism library that enables learning o...
11/16/2018 ∙ by Yanping Huang, et al. ∙ 0 ∙ share

read it

Quoc V. Le

Featured Co-authors