b'Barret Zoph'

research

∙ 05/24/2023

Flan-MoE: Scaling Instruction-Finetuned Language Models with Sparse Mixture of Experts

The explosive growth of language models and their applications have led ...

0 Sheng Shen, et al. ∙

research

∙ 05/22/2023

A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, Toxicity

Pretraining is the preliminary and fundamental step in developing capabl...

0 Shayne Longpre, et al. ∙

research

∙ 01/31/2023

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

We study the design decisions of publicly available instruction tuning m...

0 Shayne Longpre, et al. ∙

research

∙ 09/04/2022

A Review of Sparse Expert Models in Deep Learning

Sparse expert models are a thirty-year old concept re-emerging as a popu...

42 William Fedus, et al. ∙

research

∙ 04/05/2022

PaLM: Scaling Language Modeling with Pathways

Large language models have been shown to achieve remarkable performance ...

6 Aakanksha Chowdhery, et al. ∙

research

∙ 02/17/2022

Designing Effective Sparse Expert Models

Scale has opened new frontiers in natural language processing – but at a...

0 Barret Zoph, et al. ∙

research

∙ 12/13/2021

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

Scaling language models with more data, compute and parameters has drive...

4 Nan Du, et al. ∙

research

∙ 08/25/2021

Multi-Task Self-Training for Learning General Representations

Despite the fast progress in training specialized models for various tas...

0 Golnaz Ghiasi, et al. ∙

research

∙ 06/30/2021

Simple Training Strategies and Model Scaling for Object Detection

The speed-accuracy Pareto curve of object detection systems have advance...

0 Xianzhi Du, et al. ∙

research

∙ 03/13/2021

Revisiting ResNets: Improved Training and Scaling Strategies

Novel computer vision architectures monopolize the spotlight, but the im...

0 Irwan Bello, et al. ∙

research

∙ 01/11/2021

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

In deep learning, models typically reuse the same parameters for all inp...

0 William Fedus, et al. ∙

research

∙ 12/13/2020

Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation

Building instance segmentation models that are data-efficient and can ha...

18 Golnaz Ghiasi, et al. ∙

research

∙ 10/15/2020

Does Data Augmentation Benefit from Split BatchNorms

Data augmentation has emerged as a powerful technique for improving the ...

21 Amil Merchant, et al. ∙

research

∙ 06/11/2020

Rethinking Pre-training and Self-training

Pre-training is a dominant paradigm in computer vision. For example, sup...

0 Barret Zoph, et al. ∙

research

∙ 05/20/2020

Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation

Supervised learning in large discriminative models is a mainstay for mod...

14 Liang-Chieh Chen, et al. ∙

research

∙ 05/20/2020

Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation

Supervised learning in large discriminative models is a mainstay for mod...

7 Liang-Chieh Chen, et al. ∙

research

∙ 04/02/2020

Improving 3D Object Detection through Progressive Population Based Augmentation

Data augmentation has been widely adopted for object detection in 3D poi...

1 Shuyang Cheng, et al. ∙

research

∙ 12/05/2019

AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty

Modern deep neural networks can achieve high accuracy when the training ...

21 Dan Hendrycks, et al. ∙

research

∙ 09/30/2019

RandAugment: Practical data augmentation with no separate search

Recent work has shown that data augmentation has the potential to signif...

26 Ekin D. Cubuk, et al. ∙

research

∙ 06/26/2019

Learning Data Augmentation Strategies for Object Detection

Data augmentation is a critical component of training deep learning mode...

4 Barret Zoph, et al. ∙

research

∙ 04/22/2019

Attention Augmented Convolutional Networks

Convolutional networks have been the paradigm of choice in many computer...

30 Irwan Bello, et al. ∙

research

∙ 04/18/2019

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

We present SpecAugment, a simple data augmentation method for speech rec...

16 Daniel S. Park, et al. ∙

research

∙ 09/11/2018

Searching for Efficient Multi-Scale Architectures for Dense Image Prediction

The design of neural network architectures is an important component for...

2 Liang-Chieh Chen, et al. ∙

research

∙ 08/08/2018

Backprop Evolution

The back-propagation algorithm is the cornerstone of deep learning. Desp...

4 Maximilian Alber, et al. ∙

research

∙ 05/24/2018

AutoAugment: Learning Augmentation Policies from Data

In this paper, we take a closer look at data augmentation for images, an...

0 Ekin D. Cubuk, et al. ∙

research

∙ 02/09/2018

Efficient Neural Architecture Search via Parameter Sharing

We propose Efficient Neural Architecture Search (ENAS), a fast and inexp...

0 Hieu Pham, et al. ∙

research

∙ 02/09/2018

Efficient Neural Architecture Search via Parameters Sharing

We propose Efficient Neural Architecture Search (ENAS), a fast and inexp...

0 Hieu Pham, et al. ∙

research

∙ 12/02/2017

Progressive Neural Architecture Search

We propose a method for learning CNN structures that is more efficient t...

0 Chenxi Liu, et al. ∙

research

∙ 11/08/2017

Intriguing Properties of Adversarial Examples

It is becoming increasingly clear that many machine learning classifiers...

0 Ekin D. Cubuk, et al. ∙

research

∙ 10/16/2017

Searching for Activation Functions

The choice of activation functions in deep networks has a significant ef...

0 Prajit Ramachandran, et al. ∙

research

∙ 09/21/2017

Neural Optimizer Search with Reinforcement Learning

We present an approach to automate the process of discovering optimizati...

0 Irwan Bello, et al. ∙

research

∙ 07/21/2017

Learning Transferable Architectures for Scalable Image Recognition

Developing neural network image classification models often requires sig...

0 Barret Zoph, et al. ∙

research

∙ 11/05/2016

Neural Architecture Search with Reinforcement Learning

Neural networks are powerful and flexible models that work well for many...

0 Barret Zoph, et al. ∙

research

∙ 04/08/2016

Transfer Learning for Low-Resource Neural Machine Translation

The encoder-decoder framework for neural machine translation (NMT) has b...

0 Barret Zoph, et al. ∙

research

∙ 01/05/2016

Multi-Source Neural Translation

We build a multi-source machine translation model and train it to maximi...

0 Barret Zoph, et al. ∙

Barret Zoph

Featured Co-authors