Composition of Saliency Metrics for Channel Pruning with a Myopic Oracle

04/03/2020
by   Kaveena Persand, et al.
0

The computation and memory needed for Convolutional Neural Network (CNN) inference can be reduced by pruning weights from the trained network. Pruning is guided by a pruning saliency, which heuristically approximates the change in the loss function associated with the removal of specific weights. Many pruning signals have been proposed, but the performance of each heuristic depends on the particular trained network. This leaves the data scientist with a difficult choice. When using any one saliency metric for the entire pruning process, we run the risk of the metric assumptions being invalidated, leading to poor decisions being made by the metric. Ideally we could combine the best aspects of different saliency metrics. However, despite an extensive literature review, we are unable to find any prior work on composing different saliency metrics. The chief difficulty lies in combining the numerical output of different saliency metrics, which are not directly comparable. We propose a method to compose several primitive pruning saliencies, to exploit the cases where each saliency measure does well. Our experiments show that the composition of saliencies avoids many poor pruning choices identified by individual saliencies. In most cases our method finds better selections than even the best individual pruning saliency.

READ FULL TEXT
research
05/04/2022

Domino Saliency Metrics: Improving Existing Channel Saliency Metrics with Structural Information

Channel pruning is used to reduce the number of weights in a Convolution...
research
10/20/2019

Self-Adaptive Network Pruning

Deep convolutional neural networks have been proved successful on a wide...
research
03/04/2023

Visual Saliency-Guided Channel Pruning for Deep Visual Detectors in Autonomous Driving

Deep neural network (DNN) pruning has become a de facto component for de...
research
04/12/2016

What do different evaluation metrics tell us about saliency models?

How best to evaluate a saliency model's ability to predict where humans ...
research
09/07/2022

Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps

Convolutional Neural Networks (CNNs) compression is crucial to deploying...
research
05/08/2021

EZCrop: Energy-Zoned Channels for Robust Output Pruning

Recent results have revealed an interesting observation in a trained con...
research
03/12/2020

SASL: Saliency-Adaptive Sparsity Learning for Neural Network Acceleration

Accelerating the inference speed of CNNs is critical to their deployment...

Please sign up or login with your details

Forgot password? Click here to reset