Learning Invariances in Neural Networks

10/22/2020
by   Gregory Benton, et al.
0

Invariances to translations have imbued convolutional neural networks with powerful generalization properties. However, we often do not know a priori what invariances are present in the data, or to what extent a model should be invariant to a given symmetry group. We show how to learn invariances and equivariances by parameterizing a distribution over augmentations and optimizing the training loss simultaneously with respect to the network parameters and augmentation parameters. With this simple procedure we can recover the correct set and extent of invariances on image classification, regression, segmentation, and molecular property prediction from a large space of augmentations, on training data alone.

READ FULL TEXT
research
07/25/2019

Invariance reduces Variance: Understanding Data Augmentation in Deep Learning and Beyond

Many complex deep learning models have found success by exploiting symme...
research
12/19/2022

VC dimensions of group convolutional neural networks

We study the generalization capacity of group convolutional neural netwo...
research
02/01/2023

Generative Adversarial Symmetry Discovery

Despite the success of equivariant neural networks in scientific applica...
research
07/04/2021

Generalisation in Neural Networks Does not Require Feature Overlap

That shared features between train and test data are required for genera...
research
04/15/2023

Icospherical Chemical Objects (ICOs) allow for chemical data augmentation and maintain rotational, translation and permutation invariance

Dataset augmentation is a common way to deal with small datasets; Chemis...
research
08/19/2020

Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties

Equivariant neural networks (ENNs) are graph neural networks embedded in...
research
10/07/2019

Generating valid Euclidean distance matrices

Generating point clouds, e.g., molecular structures, in arbitrary rotati...

Please sign up or login with your details

Forgot password? Click here to reset