Sacrificing Accuracy for Reduced Computation: Cascaded Inference Based on Softmax Confidence

05/28/2018
by   Konstantin Berestizshevsky, et al.
0

We study the tradeoff between computational effort and accuracy in a cascade of deep neural networks. During inference, early termination in the cascade is controlled by confidence levels derived directly from the softmax outputs of intermediate classifiers. The advantage of early termination is that classification is performed using less computation, thus adjusting the computational effort to the complexity of the input. Moreover, dynamic modification of confidence thresholds allow one to trade accuracy for computational effort without requiring retraining. Basing of early termination on softmax classifier outputs is justified by experimentation that demonstrates an almost linear relation between confidence levels in intermediate classifiers and accuracy. Our experimentation with architectures based on ResNet obtained the following results. (i) A speedup of 1.5 that sacrifices 1.4 respect to the CIFAR-10 test set. (ii) A speedup of 1.19 that sacrifices 0.7 accuracy with respect to the CIFAR-100 test set. (iii) A speedup of 2.16 that sacrifices 1.4

READ FULL TEXT

page 1

page 2

page 3

page 4

05/07/2021

BasisNet: Two-stage Model Synthesis for Efficient Inference

In this work, we present BasisNet which combines recent advancements in ...
05/25/2017

Real-Time Background Subtraction Using Adaptive Sampling and Cascade of Gaussians

Background-Foreground classification is a fundamental well-studied probl...
06/15/2022

Resource-Constrained Edge AI with Early Exit Prediction

By leveraging the data sample diversity, the early-exit network recently...
03/19/2020

Overinterpretation reveals image classification model pathologies

Image classifiers are typically scored on their test set accuracy, but h...
07/21/2017

Confidence estimation in Deep Neural networks via density modelling

State-of-the-art Deep Neural Networks can be easily fooled into providin...
07/19/2018

Improving Simple Models with Confidence Profiles

In this paper, we propose a new method called ProfWeight for transferrin...
10/30/2014

Training for Fast Sequential Prediction Using Dynamic Feature Selection

We present paired learning and inference algorithms for significantly re...