On Network Design Spaces for Visual Recognition

05/30/2019
by   Ilija Radosavovic, et al.
16

Over the past several years progress in designing better neural network architectures for visual recognition has been substantial. To help sustain this rate of progress, in this work we propose to reexamine the methodology for comparing network architectures. In particular, we introduce a new comparison paradigm of distribution estimates, in which network design spaces are compared by applying statistical techniques to populations of sampled models, while controlling for confounding factors like network complexity. Compared to current methodologies of comparing point and curve estimates of model families, distribution estimates paint a more complete picture of the entire design landscape. As a case study, we examine design spaces used in neural architecture search (NAS). We find significant statistical differences between recent NAS design space variants that have been largely overlooked. Furthermore, our analysis reveals that the design spaces for standard model families like ResNeXt can be comparable to the more complex ones used in recent NAS work. We hope these insights into distribution analysis will enable more robust progress toward discovering better networks for visual recognition.

READ FULL TEXT

page 1

page 2

page 3

page 4

page 6

page 8

page 10

page 11

research
10/25/2019

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search

Neural Architecture Search (NAS) has seen an explosion of research in th...
research
03/30/2020

Designing Network Design Spaces

In this work, we present a new network design paradigm. Our goal is to h...
research
05/22/2020

An Introduction to Neural Architecture Search for Convolutional Networks

Neural Architecture Search (NAS) is a research field concerned with util...
research
11/03/2022

Towards Discovering Neural Architectures from Scratch

The discovery of neural architectures from scratch is the long-standing ...
research
11/28/2022

GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models

Neural architectures can be naturally viewed as computational graphs. Mo...
research
09/15/2021

Neural Architecture Search in operational context: a remote sensing case-study

Deep learning has become in recent years a cornerstone tool fueling key ...
research
04/02/2019

Exploring Randomly Wired Neural Networks for Image Recognition

Neural networks for image recognition have evolved through extensive man...

Please sign up or login with your details

Forgot password? Click here to reset