DeepAI AI Chat
Log In Sign Up

Powering One-shot Topological NAS with Stabilized Share-parameter Proxy

by   Ronghao Guo, et al.
Beihang University

One-shot NAS method has attracted much interest from the research community due to its remarkable training efficiency and capacity to discover high performance models. However, the search spaces of previous one-shot based works usually relied on hand-craft design and were short for flexibility on the network topology. In this work, we try to enhance the one-shot NAS by exploring high-performing network architectures in our large-scale Topology Augmented Search Space (i.e., over 3.4*10^10 different topological structures). Specifically, the difficulties for architecture searching in such a complex space has been eliminated by the proposed stabilized share-parameter proxy, which employs Stochastic Gradient Langevin Dynamics to enable fast shared parameter sampling, so as to achieve stabilized measurement of architecture performance even in search space with complex topological structures. The proposed method, namely Stablized Topological Neural Architecture Search (ST-NAS), achieves state-of-the-art performance under Multiply-Adds (MAdds) constraint on ImageNet. Our lite model ST-NAS-A achieves 76.4 with only 326M MAdds. Our moderate model ST-NAS-B achieves 77.9 just required 503M MAdds. Both of our models offer superior performances in comparison to other concurrent works on one-shot NAS.


page 1

page 2

page 3

page 4


Improving One-shot NAS by Suppressing the Posterior Fading

There is a growing interest in automated neural architecture search (NAS...

ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients

Neural Architecture Search (NAS) is widely used to automatically design ...

DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation

Neural Architecture Search (NAS) has shown great potentials in automatic...

NATS-Bench: Benchmarking NAS algorithms for Architecture Topology and Size

Neural architecture search (NAS) has attracted a lot of attention and ha...

Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities

Convolutional neural networks (CNNs) are vulnerable to adversarial examp...

ScaleNAS: One-Shot Learning of Scale-Aware Representations for Visual Recognition

Scale variance among different sizes of body parts and objects is a chal...

Continuous Ant-Based Neural Topology Search

This work introduces a novel, nature-inspired neural architecture search...