research
∙
05/24/2023
Flan-MoE: Scaling Instruction-Finetuned Language Models with Sparse Mixture of Experts
The explosive growth of language models and their applications have led ...
share
research
∙
05/22/2023
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, Toxicity
Pretraining is the preliminary and fundamental step in developing capabl...
research
∙
01/31/2023
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
We study the design decisions of publicly available instruction tuning m...
research
∙
09/04/2022
A Review of Sparse Expert Models in Deep Learning
Sparse expert models are a thirty-year old concept re-emerging as a popu...
research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
research
∙
02/17/2022
Designing Effective Sparse Expert Models
Scale has opened new frontiers in natural language processing – but at a...
research
∙
12/13/2021
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has drive...
research
∙
08/25/2021
Multi-Task Self-Training for Learning General Representations
Despite the fast progress in training specialized models for various tas...
research
∙
06/30/2021
Simple Training Strategies and Model Scaling for Object Detection
The speed-accuracy Pareto curve of object detection systems have advance...
research
∙
03/13/2021
Revisiting ResNets: Improved Training and Scaling Strategies
Novel computer vision architectures monopolize the spotlight, but the im...
research
∙
01/11/2021
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
In deep learning, models typically reuse the same parameters for all inp...
research
∙
12/13/2020
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation
Building instance segmentation models that are data-efficient and can ha...
research
∙
10/15/2020
Does Data Augmentation Benefit from Split BatchNorms
Data augmentation has emerged as a powerful technique for improving the ...
research
∙
06/11/2020
Rethinking Pre-training and Self-training
Pre-training is a dominant paradigm in computer vision. For example, sup...
research
∙
05/20/2020
Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Supervised learning in large discriminative models is a mainstay for mod...
research
∙
05/20/2020
Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
Supervised learning in large discriminative models is a mainstay for mod...
research
∙
04/02/2020
Improving 3D Object Detection through Progressive Population Based Augmentation
Data augmentation has been widely adopted for object detection in 3D poi...
research
∙
12/05/2019
AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
Modern deep neural networks can achieve high accuracy when the training ...
research
∙
09/30/2019
RandAugment: Practical data augmentation with no separate search
Recent work has shown that data augmentation has the potential to signif...
research
∙
06/26/2019
Learning Data Augmentation Strategies for Object Detection
Data augmentation is a critical component of training deep learning mode...
research
∙
04/22/2019
Attention Augmented Convolutional Networks
Convolutional networks have been the paradigm of choice in many computer...
research
∙
04/18/2019
SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition
We present SpecAugment, a simple data augmentation method for speech rec...
research
∙
09/11/2018
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
The design of neural network architectures is an important component for...
research
∙
08/08/2018
Backprop Evolution
The back-propagation algorithm is the cornerstone of deep learning. Desp...
research
∙
05/24/2018
AutoAugment: Learning Augmentation Policies from Data
In this paper, we take a closer look at data augmentation for images, an...
research
∙
02/09/2018
Efficient Neural Architecture Search via Parameter Sharing
We propose Efficient Neural Architecture Search (ENAS), a fast and inexp...
research
∙
02/09/2018
Efficient Neural Architecture Search via Parameters Sharing
We propose Efficient Neural Architecture Search (ENAS), a fast and inexp...
research
∙
12/02/2017
Progressive Neural Architecture Search
We propose a method for learning CNN structures that is more efficient t...
research
∙
11/08/2017
Intriguing Properties of Adversarial Examples
It is becoming increasingly clear that many machine learning classifiers...
research
∙
10/16/2017
Searching for Activation Functions
The choice of activation functions in deep networks has a significant ef...
research
∙
09/21/2017
Neural Optimizer Search with Reinforcement Learning
We present an approach to automate the process of discovering optimizati...
research
∙
07/21/2017
Learning Transferable Architectures for Scalable Image Recognition
Developing neural network image classification models often requires sig...
research
∙
11/05/2016
Neural Architecture Search with Reinforcement Learning
Neural networks are powerful and flexible models that work well for many...
research
∙
04/08/2016
Transfer Learning for Low-Resource Neural Machine Translation
The encoder-decoder framework for neural machine translation (NMT) has b...
research
∙
01/05/2016