Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations

02/22/2022
by   Alexander Immer, et al.
58

Data augmentation is commonly applied to improve performance of deep learning by enforcing the knowledge that certain transformations on the input preserve the output. Currently, the correct data augmentation is chosen by human effort and costly cross-validation, which makes it cumbersome to apply to new datasets. We develop a convenient gradient-based method for selecting the data augmentation. Our approach relies on phrasing data augmentation as an invariance in the prior distribution and learning it using Bayesian model selection, which has been shown to work in Gaussian processes, but not yet for deep neural networks. We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective, which can be optimised without human supervision or validation data. We show that our method can successfully recover invariances present in the data, and that this improves generalisation on image datasets.

READ FULL TEXT

page 1

page 3

research
06/14/2021

Last Layer Marginal Likelihood for Invariance Learning

Data augmentation is often used to incorporate inductive biases into mod...
research
02/04/2022

Deep invariant networks with differentiable augmentation layers

Designing learning systems which are invariant to certain data transform...
research
06/11/2019

Learning robust visual representations using data augmentation invariance

Deep convolutional neural networks trained for image object categorizati...
research
05/29/2022

Saliency Map Based Data Augmentation

Data augmentation is a commonly applied technique with two seemingly rel...
research
11/16/2019

Faster AutoAugment: Learning Augmentation Strategies using Backpropagation

Data augmentation methods are indispensable heuristics to boost the perf...
research
06/25/2021

CADDA: Class-wise Automatic Differentiable Data Augmentation for EEG Signals

Data augmentation is a key element of deep learning pipelines, as it inf...
research
02/07/2022

SODA: Self-organizing data augmentation in deep neural networks – Application to biomedical image segmentation tasks

In practice, data augmentation is assigned a predefined budget in terms ...

Please sign up or login with your details

Forgot password? Click here to reset