Simultaneously Learning Architectures and Features of Deep Neural Networks

06/11/2019
by   Tinghuai Wang, et al.
0

This paper presents a novel method which simultaneously learns the number of filters and network features repeatedly over multiple epochs. We propose a novel pruning loss to explicitly enforces the optimizer to focus on promising candidate filters while suppressing contributions of less relevant ones. In the meanwhile, we further propose to enforce the diversities between filters and this diversity-based regularization term improves the trade-off between model sizes and accuracies. It turns out the interplay between architecture and feature optimizations improves the final compressed models, and the proposed method is compared favorably to existing methods, in terms of both models sizes and accuracies for a wide range of applications including image classification, image compression and audio classification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/11/2022

Regularization-based Pruning of Irrelevant Weights in Deep Neural Architectures

Deep neural networks exploiting millions of parameters are nowadays the ...
research
11/16/2021

Neuron-based Pruning of Deep Neural Networks with Better Generalization using Kronecker Factored Curvature Approximation

Existing methods of pruning deep neural networks focus on removing unnec...
research
01/15/2020

A "Network Pruning Network" Approach to Deep Model Compression

We present a filter pruning approach for deep model compression, using a...
research
09/23/2020

Pruning Convolutional Filters using Batch Bridgeout

State-of-the-art computer vision models are rapidly increasing in capaci...
research
11/18/2021

Automatic Neural Network Pruning that Efficiently Preserves the Model Accuracy

Neural networks performance has been significantly improved in the last ...
research
10/05/2021

Deep Optimization of Parametric IIR Filters for Audio Equalization

This paper describes a novel Deep Learning method for the design of IIR ...
research
06/01/2011

Identifying Mislabeled Training Data

This paper presents a new approach to identifying and eliminating mislab...

Please sign up or login with your details

Forgot password? Click here to reset