Enhanced Gradient for Differentiable Architecture Search

03/23/2021
by   Haichao Zhang, et al.
0

In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve network performance (e.g., classification accuracy) and reduce network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a relaxation method based on the gradient is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, we apply an evolutionary multi-objective algorithm to complete the automatic design from blocks to the target network. The experiment results demonstrate that our method outperforms all evaluated hand-crafted networks in image classification, with an error rate of on CIFAR10 and an error rate of on CIFAR100, both at network parameter size less than one megabit. Moreover, compared with other neural architecture search methods, our method offers a tremendous reduction in designed network architecture parameters.

READ FULL TEXT

page 1

page 13

research
11/19/2020

Effective, Efficient and Robust Neural Architecture Search

Recent advances in adversarial attacks show the vulnerability of deep ne...
research
06/16/2020

Fine-Tuning DARTS for Image Classification

Neural Architecture Search (NAS) has gained attraction due to superior c...
research
11/16/2019

Automatic Design of CNNs via Differentiable Neural Architecture Search for PolSAR Image Classification

Convolutional neural networks (CNNs) have shown good performance in pola...
research
11/16/2019

Optimized CNN for PolSAR Image Classification via Differentiable Neural Architecture Search

Convolutional neural networks (CNNs) realize the automation of feature e...
research
07/22/2019

MemNet: Memory-Efficiency Guided Neural Architecture Search with Augment-Trim learning

Recent studies on automatic neural architectures search have demonstrate...
research
07/12/2020

VINNAS: Variational Inference-based Neural Network Architecture Search

In recent years, neural architecture search (NAS) has received intensive...
research
11/26/2018

GP-CNAS: Convolutional Neural Network Architecture Search with Genetic Programming

Convolutional neural networks (CNNs) are effective at solving difficult ...

Please sign up or login with your details

Forgot password? Click here to reset