A scalable convolutional neural network for task-specified scenarios via knowledge distillation

09/19/2016
by   Mengnan Shi, et al.
0

In this paper, we explore the redundancy in convolutional neural network, which scales with the complexity of vision tasks. Considering that many front-end visual systems are interested in only a limited range of visual targets, the removing of task-specified network redundancy can promote a wide range of potential applications. We propose a task-specified knowledge distillation algorithm to derive a simplified model with pre-set computation cost and minimized accuracy loss, which suits the resource constraint front-end systems well. Experiments on the MNIST and CIFAR10 datasets demonstrate the feasibility of the proposed approach as well as the existence of task-specified redundancy.

READ FULL TEXT
research
12/17/2020

Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixup

Knowledge distillation, which involves extracting the "dark knowledge" f...
research
10/19/2021

Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation

Knowledge Distillation is becoming one of the primary trends among neura...
research
11/11/2020

Efficient Knowledge Distillation for RNN-Transducer Models

Knowledge Distillation is an effective method of transferring knowledge ...
research
11/16/2022

Yield Evaluation of Citrus Fruits based on the YoloV5 compressed by Knowledge Distillation

In the field of planting fruit trees, pre-harvest estimation of fruit yi...
research
05/17/2019

Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

Convolutional neural networks have been widely deployed in various appli...
research
12/17/2019

Joint Architecture and Knowledge Distillation in Convolutional Neural Network for Offline Handwritten Chinese Text Recognition

The technique of distillation helps transform cumbersome neural network ...
research
03/25/2022

CNN LEGO: Disassembling and Assembling Convolutional Neural Network

Convolutional Neural Network (CNN), which mimics human visual perception...

Please sign up or login with your details

Forgot password? Click here to reset