Architectural Adversarial Robustness: The Case for Deep Pursuit

11/29/2020
by   George Cazenavette, et al.
0

Despite their unmatched performance, deep neural networks remain susceptible to targeted attacks by nearly imperceptible levels of adversarial noise. While the underlying cause of this sensitivity is not well understood, theoretical analyses can be simplified by reframing each layer of a feed-forward network as an approximate solution to a sparse coding problem. Iterative solutions using basis pursuit are theoretically more stable and have improved adversarial robustness. However, cascading layer-wise pursuit implementations suffer from error accumulation in deeper networks. In contrast, our new method of deep pursuit approximates the activations of all layers as a single global optimization problem, allowing us to consider deeper, real-world architectures with skip connections such as residual networks. Experimentally, our approach demonstrates improved robustness to adversarial noise.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/02/2018

On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks

Parsimonious representations in data modeling are ubiquitous and central...
research
06/18/2021

Residual Error: a New Performance Measure for Adversarial Robustness

Despite the significant advances in deep learning over the past decade, ...
research
05/02/2019

Weight Map Layer for Noise and Adversarial Attack Robustness

Convolutional neural networks (CNNs) are known for their good performanc...
research
03/23/2020

Architectural Resilience to Foreground-and-Background Adversarial Noise

Adversarial attacks in the form of imperceptible perturbations of normal...
research
03/10/2021

Reframing Neural Networks: Deep Structure in Overcomplete Representations

In comparison to classical shallow representation learning techniques, d...
research
03/28/2022

WSEBP: A Novel Width-depth Synchronous Extension-based Basis Pursuit Algorithm for Multi-Layer Convolutional Sparse Coding

The pursuit algorithms integrated in multi-layer convolutional sparse co...
research
05/12/2023

Optimal signal propagation in ResNets through residual scaling

Residual networks (ResNets) have significantly better trainability and t...

Please sign up or login with your details

Forgot password? Click here to reset