S2DNAS: Transforming Static CNN Model for Dynamic Inference via Neural Architecture Search

11/16/2019
by   Zhihang Yuan, et al.
0

Recently, dynamic inference has emerged as a promising way to reduce the computational cost of deep convolutional neural networks (CNNs). In contrast to static methods (e.g., weight pruning), dynamic inference adaptively adjusts the inference process according to each input sample, which can considerably reduce the computational cost on "easy" samples while maintaining the overall model performance. In this paper, we introduce a general framework, S2DNAS, which can transform various static CNN models to support dynamic inference via neural architecture search. To this end, based on a given CNN model, we first generate a CNN architecture space in which each architecture is a multi-stage CNN generated from the given model using some predefined transformations. Then, we propose a reinforcement learning based approach to automatically search for the optimal CNN architecture in the generated space. At last, with the searched multi-stage network, we can perform dynamic inference by adaptively choosing a stage to evaluate for each sample. Unlike previous works that introduce irregular computations or complex controllers in the inference or re-design a CNN model from scratch, our method can generalize to most of the popular CNN architectures and the searched dynamic network can be directly deployed using existing deep learning frameworks in various hardware devices.

READ FULL TEXT

page 3

page 4

research
09/19/2020

ENAS4D: Efficient Multi-stage CNN Architecture Search for Dynamic Inference

Dynamic inference is a feasible way to reduce the computational cost of ...
research
09/11/2020

Optimizing Convolutional Neural Network Architecture via Information Field

CNN architecture design has attracted tremendous attention of improving ...
research
11/10/2022

Cherry Hypothesis: Identifying the Cherry on the Cake for Dynamic Networks

Dynamic networks have been extensively explored as they can considerably...
research
04/10/2020

ModuleNet: Knowledge-inherited Neural Architecture Search

Although Neural Architecture Search (NAS) can bring improvement to deep ...
research
11/16/2018

Stochastic Adaptive Neural Architecture Search for Keyword Spotting

The problem of keyword spotting i.e. identifying keywords in a real-time...
research
05/28/2019

SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers

The vast majority of processors in the world are actually microcontrolle...
research
12/13/2019

PreVIous: A Methodology for Prediction of Visual Inference Performance on IoT Devices

This paper presents PreVIous, a methodology to predict the performance o...

Please sign up or login with your details

Forgot password? Click here to reset