Doubly Nested Network for Resource-Efficient Inference

06/20/2018
by   Jaehong Kim, et al.
0

We propose doubly nested network(DNNet) where all neurons represent their own sub-models that solve the same task. Every sub-model is nested both layer-wise and channel-wise. While nesting sub-models layer-wise is straight-forward with deep-supervision as proposed in xie2015holistically, channel-wise nesting has not been explored in the literature to our best knowledge. Channel-wise nesting is non-trivial as neurons between consecutive layers are all connected to each other. In this work, we introduce a technique to solve this problem by sorting channels topologically and connecting neurons accordingly. For the purpose, channel-causal convolutions are used. Slicing doubly nested network gives a working sub-network. The most notable application of our proposed network structure with slicing operation is resource-efficient inference. At test time, computing resources such as time and memory available for running the prediction algorithm can significantly vary across devices and applications. Given a budget constraint, we can slice the network accordingly and use a sub-model for inference within budget, requiring no additional computation such as training or fine-tuning after deployment. We demonstrate the effectiveness of our approach in several practical scenarios of utilizing available resource efficiently.

READ FULL TEXT
research
04/03/2019

Model Slicing for Supporting Complex Analytics with Elastic Inference Cost and Resource Constraints

Deep learning models have been used to support analytics beyond simple a...
research
04/10/2023

ADS_UNet: A Nested UNet for Histopathology Image Segmentation

The UNet model consists of fully convolutional network (FCN) layers arra...
research
12/11/2017

Learning Nested Sparse Structures in Deep Neural Networks

Recently, there have been increasing demands to construct compact deep a...
research
05/02/2023

Distill or Annotate? Cost-Efficient Fine-Tuning of Compact Models

Fine-tuning large models is highly effective, however, inference using t...
research
03/25/2017

More is Less: A More Complicated Network with Less Inference Complexity

In this paper, we present a novel and general network structure towards ...
research
03/07/2022

Dynamic ConvNets on Tiny Devices via Nested Sparsity

This work introduces a new training and compression pipeline to build Ne...

Please sign up or login with your details

Forgot password? Click here to reset