AI Chat AI Image Generator AI Video Text to Speech

SGAD: Soft-Guided Adaptively-Dropped Neural Network

07/04/2018

∙

by Zhisheng Wang, et al.

∙

∙

Deep neural networks (DNNs) have been proven to have many redundancies. Hence, many efforts have been made to compress DNNs. However, the existing model compression methods treat all the input samples equally while ignoring the fact that the difficulties of various input samples being correctly classified are different. To address this problem, DNNs with adaptive dropping mechanism are well explored in this work. To inform the DNNs how difficult the input samples can be classified, a guideline that contains the information of input samples is introduced to improve the performance. Based on the developed guideline and adaptive dropping mechanism, an innovative soft-guided adaptively-dropped (SGAD) neural network is proposed in this paper. Compared with the 32 layers residual neural networks, the presented SGAD can reduce the FLOPs by 77

Zhisheng Wang
8 publications
Fangxuan Sun
2 publications
Jun Lin
17 publications
Zhongfeng Wang
22 publications
Bo Yuan
72 publications

page 1

page 2

page 3

page 4

research

∙ 07/07/2020

Soft Labeling Affects Out-of-Distribution Detection of Deep Neural Networks

Soft labeling becomes a common output regularization for generalization ...

0 Doyup Lee, et al. ∙

research

∙ 03/16/2022

Reducing Flipping Errors in Deep Neural Networks

Deep neural networks (DNNs) have been widely applied in various domains ...

1 Xiang Deng, et al. ∙

research

∙ 02/25/2017

Adaptive Neural Networks for Efficient Inference

We present an approach to adaptively utilize deep neural networks in ord...

0 Tolga Bolukbasi, et al. ∙

research

∙ 09/29/2018

To compress or not to compress: Understanding the Interactions between Adversarial Attacks and Neural Network Compression

As deep neural networks (DNNs) become widely used, pruned and quantised ...

0 Yiren Zhao, et al. ∙

research

∙ 11/15/2019

ASCAI: Adaptive Sampling for acquiring Compact AI

This paper introduces ASCAI, a novel adaptive sampling methodology that ...

0 Mojan Javaheripi, et al. ∙

research

∙ 02/15/2017

An Analysis of Ability in Deep Neural Networks

Deep neural networks (DNNs) have made significant progress in a number o...

0 John P. Lalor, et al. ∙

research

∙ 08/17/2023

Dynamic Neural Network is All You Need: Understanding the Robustness of Dynamic Mechanisms in Neural Networks

Deep Neural Networks (DNNs) have been used to solve different day-to-day...

0 Mirazul Haque, et al. ∙