Fast Neural Network Adaptation via Parameter Remapping and Architecture Search

01/08/2020
by   Jiemin Fang, et al.
26

Deep neural networks achieve remarkable performance in many computer vision tasks. Most state-of-the-art (SOTA) semantic segmentation and object detection approaches reuse neural network architectures designed for image classification as the backbone, commonly pre-trained on ImageNet. However, performance gains can be achieved by designing network architectures specifically for detection and segmentation, as shown by recent neural architecture search (NAS) research for detection and segmentation. One major challenge though, is that ImageNet pre-training of the search space representation (a.k.a. super network) or the searched networks incurs huge computational cost. In this paper, we propose a Fast Neural Network Adaptation (FNA) method, which can adapt both the architecture and parameters of a seed network (e.g. a high performing manually designed backbone) to become a network with different depth, width, or kernels via a Parameter Remapping technique, making it possible to utilize NAS for detection/segmentation tasks a lot more efficiently. In our experiments, we conduct FNA on MobileNetV2 to obtain new networks for both segmentation and detection that clearly out-perform existing networks designed both manually and by NAS. The total computation cost of FNA is significantly less than SOTA segmentation/detection NAS approaches: 1737× less than DPC, 6.8× less than Auto-DeepLab and 7.4× less than DetNAS. The code is available at https://github.com/JaminFong/FNA.

READ FULL TEXT

page 13

page 14

research
06/21/2020

FNA++: Fast Network Adaptation via Parameter Remapping and Architecture Search

Deep neural networks achieve remarkable performance in many computer vis...
research
06/17/2022

Neural Architecture Adaptation for Object Detection by Searching Channel Dimensions and Mapping Pre-trained Parameters

Most object detection frameworks use backbone architectures originally d...
research
03/08/2022

Evolutionary Neural Cascade Search across Supernetworks

To achieve excellent performance with modern neural networks, having the...
research
04/14/2022

HASA: Hybrid Architecture Search with Aggregation Strategy for Echinococcosis Classification and Ovary Segmentation in Ultrasound Images

Different from handcrafted features, deep neural networks can automatica...
research
07/13/2022

MRF-UNets: Searching UNet with Markov Random Fields

UNet [27] is widely used in semantic segmentation due to its simplicity ...
research
09/01/2021

Searching for Efficient Multi-Stage Vision Transformers

Vision Transformer (ViT) demonstrates that Transformer for natural langu...
research
08/13/2021

EEEA-Net: An Early Exit Evolutionary Neural Architecture Search

The goals of this research were to search for Convolutional Neural Netwo...

Please sign up or login with your details

Forgot password? Click here to reset