DeepAI AI Chat
Log In Sign Up

EfficientNetV2: Smaller Models and Faster Training

by   Mingxing Tan, et al.

This paper introduces EfficientNetV2, a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models. To develop this family of models, we use a combination of training-aware neural architecture search and scaling, to jointly optimize training speed and parameter efficiency. The models were searched from the search space enriched with new ops such as Fused-MBConv. Our experiments show that EfficientNetV2 models train much faster than state-of-the-art models while being up to 6.8x smaller. Our training can be further sped up by progressively increasing the image size during training, but it often causes a drop in accuracy. To compensate for this accuracy drop, we propose to adaptively adjust regularization (e.g., dropout and data augmentation) as well, such that we can achieve both fast training and good accuracy. With progressive learning, our EfficientNetV2 significantly outperforms previous models on ImageNet and CIFAR/Cars/Flowers datasets. By pretraining on the same ImageNet21k, our EfficientNetV2 achieves 87.3 ImageNet ILSVRC2012, outperforming the recent ViT by 2.0 training 5x-11x faster using the same computing resources. Code will be available at


page 5

page 12


Neural Architecture Search using Progressive Evolution

Vanilla neural architecture search using evolutionary algorithms (EA) in...

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

Convolutional Neural Networks (ConvNets) are commonly developed at a fix...

Can weight sharing outperform random architecture search? An investigation with TuNAS

Efficient Neural Architecture Search methods based on weight sharing hav...

TResNet: High Performance GPU-Dedicated Architecture

Many deep learning models, developed in recent years, reach higher Image...

DropNAS: Grouped Operation Dropout for Differentiable Architecture Search

Neural architecture search (NAS) has shown encouraging results in automa...

F-COREF: Fast, Accurate and Easy to Use Coreference Resolution

We introduce fastcoref, a python package for fast, accurate, and easy-to...

Augmenting Sub-model to Improve Main Model

Image classification has improved with the development of training techn...

Code Repositories


A PyTorch implementation of EfficientNet and EfficientNetV2 (coming soon!)

view repo