Improving the Accuracy of Early Exits in Multi-Exit Architectures via Curriculum Learning

04/21/2021
by   Arian Bakhtiarnia, et al.
8

Deploying deep learning services for time-sensitive and resource-constrained settings such as IoT using edge computing systems is a challenging task that requires dynamic adjustment of inference time. Multi-exit architectures allow deep neural networks to terminate their execution early in order to adhere to tight deadlines at the cost of accuracy. To mitigate this cost, in this paper we introduce a novel method called Multi-Exit Curriculum Learning that utilizes curriculum learning, a training strategy for neural networks that imitates human learning by sorting the training samples based on their difficulty and gradually introducing them to the network. Experiments on CIFAR-10 and CIFAR-100 datasets and various configurations of multi-exit architectures show that our method consistently improves the accuracy of early exits compared to the standard training approach.

READ FULL TEXT
research
05/19/2021

Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead

Deploying deep learning models in time-critical applications with limite...
research
01/03/2018

ScreenerNet: Learning Curriculum for Neural Networks

We propose to learn a curriculum or a syllabus for supervised learning w...
research
05/18/2022

LeRaC: Learning Rate Curriculum

Most curriculum learning methods require an approach to sort the data sa...
research
06/29/2021

Multi-Exit Vision Transformer for Dynamic Inference

Deep neural networks can be converted to multi-exit architectures by ins...
research
01/13/2020

Rethinking Curriculum Learning with Incremental Labels and Adaptive Compensation

Like humans, deep networks learn better when samples are organized and i...
research
11/21/2019

MSD: Multi-Self-Distillation Learning via Multi-classifiers within Deep Neural Networks

As the development of neural networks, more and more deep neural network...
research
11/05/2020

Conflicting Bundles: Adapting Architectures Towards the Improved Training of Deep Neural Networks

Designing neural network architectures is a challenging task and knowing...

Please sign up or login with your details

Forgot password? Click here to reset