Evolving Transferable Pruning Functions

10/21/2021
by   Yuchen Liu, et al.
0

Channel pruning has made major headway in the design of efficient deep learning models. Conventional approaches adopt human-made pruning functions to score channels' importance for channel pruning, which requires domain knowledge and could be sub-optimal. In this work, we propose an end-to-end framework to automatically discover strong pruning metrics. Specifically, we craft a novel design space for expressing pruning functions and leverage an evolution strategy, genetic programming, to evolve high-quality and transferable pruning functions. Unlike prior methods, our approach can not only provide compact pruned networks for efficient inference, but also novel closed-form pruning metrics that are mathematically explainable and thus generalizable to different pruning tasks. The evolution is conducted on small datasets while the learned functions are transferable to larger datasets without any manual modification. Compared to direct evolution on a large dataset, our strategy shows better cost-effectiveness. When applied to more challenging datasets, different from those used in the evolution process, e.g., ILSVRC-2012, an evolved function achieves state-of-the-art pruning results.

READ FULL TEXT
research
04/29/2020

Rethinking Class-Discrimination Based CNN Channel Pruning

Channel pruning has received ever-increasing focus on network compressio...
research
11/10/2018

Using NonBacktracking Expansion to Analyze k-core Pruning Process

We induce the NonBacktracking Expansion Branch method to analyze the k-c...
research
05/31/2022

ViNNPruner: Visual Interactive Pruning for Deep Learning

Neural networks grow vastly in size to tackle more sophisticated tasks. ...
research
12/01/2021

Bumblebee: A Path Towards Fully Autonomous Robotic Vine Pruning

Dormant season grapevine pruning requires skilled seasonal workers durin...
research
05/23/2018

AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inference

Channel pruning is an important family of methods to speedup deep model'...
research
02/25/2019

The MBPEP: a deep ensemble pruning algorithm providing high quality uncertainty prediction

Machine learning algorithms have been effectively applied into various r...
research
03/04/2023

Visual Saliency-Guided Channel Pruning for Deep Visual Detectors in Autonomous Driving

Deep neural network (DNN) pruning has become a de facto component for de...

Please sign up or login with your details

Forgot password? Click here to reset