Robustness Against Outliers For Deep Neural Networks By Gradient Conjugate Priors

05/21/2019
by   Pavel Gurevich, et al.
0

We analyze a new robust method for the reconstruction of probability distributions of observed data in the presence of output outliers. It is based on a so-called gradient conjugate prior (GCP) network which outputs the parameters of a prior. By rigorously studying the dynamics of the GCP learning process, we derive an explicit formula for correcting the obtained variance of the marginal distribution and removing the bias caused by outliers in the training set. Assuming a Gaussian (input-dependent) ground truth distribution contaminated with a proportion ε of outliers, we show that the fitted mean is in a c e^-1/ε-neighborhood of the ground truth mean and the corrected variance is in a bε-neighborhood of the ground truth variance, whereas the uncorrected variance of the marginal distribution can even be infinite. We explicitly find b as a function of the output of the GCP network, without a priori knowledge of the outliers (possibly input-dependent) distribution. Experiments with synthetic and real-world data sets indicate that the GCP network fitted with a standard optimizer outperforms other robust methods for regression.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/07/2018

Gradient conjugate priors and deep neural networks

The paper deals with learning the probability distribution of the observ...
research
07/27/2019

Bayesian Robustness: A Nonasymptotic Viewpoint

We study the problem of robustly estimating the posterior distribution f...
research
10/28/2021

Aggregation as Unsupervised Learning and its Evaluation

Regression uses supervised machine learning to find a model that combine...
research
05/04/2021

Towards Error Measures which Influence a Learners Inductive Bias to the Ground Truth

Artificial intelligence is applied in a range of sectors, and is relied ...
research
10/16/2018

Learning Two-layer Neural Networks with Symmetric Inputs

We give a new algorithm for learning a two-layer neural network under a ...
research
04/16/2019

SynC: A Unified Framework for Generating Synthetic Population with Gaussian Copula

Synthetic population generation is the process of combining multiple soc...
research
08/25/2020

Temporal Action Localization with Variance-Aware Networks

This work addresses the problem of temporal action localization with Var...

Please sign up or login with your details

Forgot password? Click here to reset