Max-Mahalanobis Linear Discriminant Analysis Networks

02/26/2018
by   Tianyu Pang, et al.
0

A deep neural network (DNN) consists of a nonlinear transformation from an input to a feature representation, followed by a common softmax linear classifier. Though many efforts have been devoted to designing a proper architecture for nonlinear transformation, little investigation has been done on the classifier part. In this paper, we show that a properly designed classifier can improve robustness to adversarial attacks and lead to better prediction results. Specifically, we define a Max-Mahalanobis distribution (MMD) and theoretically show that if the input distributes as a MMD, the linear discriminant analysis (LDA) classifier will have the best robustness to adversarial examples. We further propose a novel Max-Mahalanobis linear discriminant analysis (MM-LDA) network, which explicitly maps a complicated data distribution in the input space to a MMD in the latent feature space and then applies LDA to make predictions. Our results demonstrate that the MM-LDA networks are significantly more robust to adversarial attacks, and have better performance in class-biased classification.

READ FULL TEXT
research
06/23/2019

Nested Cavity Classifier: performance and remedy

Nested Cavity Classifier (NCC) is a classification rule that pursues par...
research
03/23/2022

Enhancing Classifier Conservativeness and Robustness by Polynomiality

We illustrate the detrimental effect, such as overconfident decisions, t...
research
09/30/2019

Influence functions for Linear Discriminant Analysis: Sensitivity analysis and efficient influence diagnostics

Whilst influence functions for linear discriminant analysis (LDA) have b...
research
09/02/2021

Two Shifts for Crop Mapping: Leveraging Aggregate Crop Statistics to Improve Satellite-based Maps in New Regions

Crop type mapping at the field level is critical for a variety of applic...
research
11/04/2020

Capped norm linear discriminant analysis and its applications

Classical linear discriminant analysis (LDA) is based on squared Frobeni...
research
05/15/2021

Regularized Deep Linear Discriminant Analysis

As a non-linear extension of the classic Linear Discriminant Analysis(LD...
research
04/06/2019

TULIP: A Toolbox for Linear Discriminant Analysis with Penalties

Linear discriminant analysis (LDA) is a powerful tool in building classi...

Please sign up or login with your details

Forgot password? Click here to reset