Active Subspace of Neural Networks: Structural Analysis and Universal Attacks

10/29/2019
by   Chunfeng Cui, et al.
33

Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability and deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of "active neurons" at each intermediate layer and reduce the number of neurons from several thousands to several dozens. This motivates us to change the network structure and to develop a new and more compact network, referred to as ASNet, that has significantly fewer model parameters. Secondly, we propose analyzing the vulnerability of a neural network using active subspace and finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability. Our experiments on CIFAR-10 show that ASNet can achieve 23.98× parameter and 7.30× flops reduction. The universal active subspace attack vector can achieve around 20 ratio compared with the existing approach in all of our numerical experiments. The PyTorch codes for this paper are available online.

READ FULL TEXT

page 11

page 18

page 19

research
09/06/2023

Learning Active Subspaces for Effective and Scalable Uncertainty Quantification in Deep Neural Networks

Bayesian inference for neural networks, or Bayesian deep learning, has t...
research
03/10/2019

Uncertainty Propagation in Deep Neural Network Using Active Subspace

The inputs of deep neural network (DNN) from real-world data usually com...
research
09/24/2019

Structural Change Analysis of Active Cryptocurrency Market

Structural Change Analysis of Active Cryptocurrency Market...
research
01/01/2021

Active Learning Under Malicious Mislabeling and Poisoning Attacks

Deep neural networks usually require large labeled datasets for training...
research
08/08/2020

A Fully Bayesian Gradient-Free Supervised Dimension Reduction Method using Gaussian Processes

Modern day engineering problems are ubiquitously characterized by sophis...
research
05/24/2020

Detecting Adversarial Examples for Speech Recognition via Uncertainty Quantification

Machine learning systems and also, specifically, automatic speech recogn...
research
06/24/2010

Active Sites model for the B-Matrix Approach

This paper continues on the work of the B-Matrix approach in hebbian lea...

Please sign up or login with your details

Forgot password? Click here to reset