DeepAI AI Chat
Log In Sign Up

A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs

11/03/2020
by   Souvik Kundu, et al.
0

This paper presents a dynamic network rewiring (DNR) method to generate pruned deep neural network (DNN) models that are robust against adversarial attacks yet maintain high accuracy on clean images. In particular, the disclosed DNR method is based on a unified constrained optimization formulation using a hybrid loss function that merges ultra-high model compression with robust adversarial training. This training strategy dynamically adjusts inter-layer connectivity based on per-layer normalized momentum computed from the hybrid loss function. In contrast to existing robust pruning frameworks that require multiple training iterations, the proposed learning strategy achieves an overall target pruning ratio with only a single training iteration and can be tuned to support both irregular and structured channel pruning. To evaluate the merits of DNR, experiments were performed with two widely accepted models, namely VGG16 and ResNet-18, on CIFAR-10, CIFAR-100 as well as with VGG16 on Tiny-ImageNet. Compared to the baseline uncompressed models, DNR provides over20x compression on all the datasets with no significant drop in either clean or adversarial classification accuracy. Moreover, our experiments show that DNR consistently finds compressed models with better clean and adversarial image classification performance than what is achievable through state-of-the-art alternatives.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/24/2020

On Pruning Adversarially Robust Neural Networks

In safety-critical but computationally resource-constrained applications...
03/29/2019

Second Rethinking of Network Pruning in the Adversarial Setting

It is well known that deep neural networks (DNNs) are vulnerable to adve...
06/15/2022

Hardening DNNs against Transfer Attacks during Network Compression using Greedy Adversarial Pruning

The prevalence and success of Deep Neural Network (DNN) applications in ...
08/11/2022

WeightMom: Learning Sparse Networks using Iterative Momentum-based pruning

Deep Neural Networks have been used in a wide variety of applications wi...
02/03/2022

Robust Binary Models by Pruning Randomly-initialized Networks

We propose ways to obtain robust models against adversarial attacks from...
06/07/2020

EDropout: Energy-Based Dropout and Pruning of Deep Neural Networks

Dropout is a well-known regularization method by sampling a sub-network ...
02/23/2021

Winning Ticket in Noisy Image Classification

Modern deep neural networks (DNNs) become frail when the datasets contai...

Code Repositories