Explicit Regularisation in Gaussian Noise Injections

07/14/2020
by   Alexander Camuto, et al.
0

We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have been few studies on understanding the regularising effect they induce when applied to network activations. Here we derive the explicit regulariser of GNIs, obtained by marginalising out the injected noise, and show that it is a form of Tikhonov regularisation which penalises functions with high-frequency components in the Fourier domain. We show analytically and empirically that such regularisation produces calibrated classifiers with large classification margins and that the explicit regulariser we derive is able to reproduce these effects.

READ FULL TEXT

page 7

page 19

research
06/11/2019

A new data fitting method for stretched Gaussian noise: stretched least square method

Stretched Gaussian distribution is the fundamental solution of the Hausd...
research
02/22/2018

Robustness of classifiers to uniform ℓ_p and Gaussian noise

We study the robustness of classifiers to various kinds of random noise ...
research
02/13/2021

Asymmetric Heavy Tails and Implicit Bias in Gaussian Noise Injections

Gaussian noise injections (GNIs) are a family of simple and widely-used ...
research
03/01/2018

Learning with Correntripy-induced Losses for Regression with Mixture of Symmetric Stable Noise

In recent years, correntropy and its applications in machine learning ha...
research
03/01/2018

Learning with Correntropy-induced Losses for Regression with Mixture of Symmetric Stable Noise

In recent years, correntropy and its applications in machine learning ha...
research
01/03/2022

Multiview point cloud registration with anisotropic and space-varying localization noise

In this paper, we address the problem of registering multiple point clou...
research
03/29/2023

Quantifying VIO Uncertainty

We compute the uncertainty of XIVO, a monocular visual-inertial odometry...

Please sign up or login with your details

Forgot password? Click here to reset