How Not to Give a FLOP: Combining Regularization and Pruning for Efficient Inference

03/30/2020
by   Tai Vu, et al.
0

The challenge of speeding up deep learning models during the deployment phase has been a large, expensive bottleneck in the modern tech industry. In this paper, we examine the use of both regularization and pruning for reduced computational complexity and more efficient inference in Deep Neural Networks (DNNs). In particular, we apply mixup and cutout regularizations and soft filter pruning to the ResNet architecture, focusing on minimizing floating point operations (FLOPs). Furthermore, by using regularization in conjunction with network pruning, we show that such a combination makes a substantial improvement over each of the two techniques individually.

READ FULL TEXT
research
10/19/2020

Softer Pruning, Incremental Regularization

Network pruning is widely used to compress Deep Neural Networks (DNNs). ...
research
12/24/2019

Pruning Deep Neural Networks Architectures with Evolution Strategy

Currently, Deep Neural Networks (DNNs) are used to solve all kinds of pr...
research
08/28/2023

A Generalization of Continuous Relaxation in Structured Pruning

Deep learning harnesses massive parallel floating-point processing to tr...
research
10/23/2018

Deep Neural Network inference with reduced word length

Deep neural networks (DNN) are powerful models for many pattern recognit...
research
06/08/2020

A Framework for Neural Network Pruning Using Gibbs Distributions

Neural network pruning is an important technique for creating efficient ...
research
05/21/2020

CPOT: Channel Pruning via Optimal Transport

Recent advances in deep neural networks (DNNs) lead to tremendously grow...
research
05/31/2022

ViNNPruner: Visual Interactive Pruning for Deep Learning

Neural networks grow vastly in size to tackle more sophisticated tasks. ...

Please sign up or login with your details

Forgot password? Click here to reset