Sparsity Meets Robustness: Channel Pruning for the Feynman-Kac Formalism Principled Robust Deep Neural Nets

by   Thu Dinh, et al.
University of California, Irvine

Deep neural nets (DNNs) compression is crucial for adaptation to mobile devices. Though many successful algorithms exist to compress naturally trained DNNs, developing efficient and stable compression algorithms for robustly trained DNNs remains widely open. In this paper, we focus on a co-design of efficient DNN compression algorithms and sparse neural architectures for robust and accurate deep learning. Such a co-design enables us to advance the goal of accommodating both sparsity and robustness. With this objective in mind, we leverage the relaxed augmented Lagrangian based algorithms to prune the weights of adversarially trained DNNs, at both structured and unstructured levels. Using a Feynman-Kac formalism principled robust and sparse DNNs, we can at least double the channel sparsity of the adversarially trained ResNet20 for CIFAR10 classification, meanwhile, improve the natural accuracy by 8.69% and the robust accuracy under the benchmark 20 iterations of IFGSM attack by 5.42%. The code is available at <>.


page 4

page 15

page 16


An Integrated Approach to Produce Robust Models with High Efficiency

Deep Neural Networks (DNNs) needs to be both efficient and robust for pr...

The Search for Sparse, Robust Neural Networks

Recent work on deep neural network pruning has shown there exist sparse ...

Graph Interpolating Activation Improves Both Natural and Robust Accuracies in Data-Efficient Deep Learning

Improving the accuracy and robustness of deep neural nets (DNNs) and ada...

RadiX-Net: Structured Sparse Matrices for Deep Neural Networks

The sizes of deep neural networks (DNNs) are rapidly outgrowing the capa...

DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks

Efficient deep neural network (DNN) models equipped with compact operato...

Multi-Objective Optimization for Sparse Deep Neural Network Training

Different conflicting optimization criteria arise naturally in various D...

Sparsity-guided Network Design for Frame Interpolation

DNN-based frame interpolation, which generates intermediate frames from ...

Please sign up or login with your details

Forgot password? Click here to reset