Nonlinearities in Steerable SO(2)-Equivariant CNNs

09/14/2021
by   Daniel Franzen, et al.
0

Invariance under symmetry is an important problem in machine learning. Our paper looks specifically at equivariant neural networks where transformations of inputs yield homomorphic transformations of outputs. Here, steerable CNNs have emerged as the standard solution. An inherent problem of steerable representations is that general nonlinear layers break equivariance, thus restricting architectural choices. Our paper applies harmonic distortion analysis to illuminate the effect of nonlinearities on Fourier representations of SO(2). We develop a novel FFT-based algorithm for computing representations of non-linearly transformed activations while maintaining band-limitation. It yields exact equivariance for polynomial (approximations of) nonlinearities, as well as approximate solutions with tunable accuracy for general functions. We apply the approach to build a fully E(3)-equivariant network for sampled 3D surface data. In experiments with 2D and 3D data, we obtain results that compare favorably to the state-of-the-art in terms of accuracy while permitting continuous symmetry and exact equivariance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/18/2019

Probabilistic symmetry and invariant neural networks

In an effort to improve the performance of deep neural networks in data-...
research
10/08/2022

Symmetry Subgroup Defense Against Adversarial Attacks

Adversarial attacks and defenses disregard the lack of invariance of con...
research
06/07/2021

Encoding Involutory Invariance in Neural Networks

In certain situations, Neural Networks (NN) are trained upon data that o...
research
04/24/2022

Learning Symmetric Embeddings for Equivariant World Models

Incorporating symmetries can lead to highly data-efficient and generaliz...
research
01/31/2023

The passive symmetries of machine learning

Any representation of data involves arbitrary investigator choices. Beca...
research
11/11/2022

Equivariance with Learned Canonicalization Functions

Symmetry-based neural networks often constrain the architecture in order...
research
03/17/2022

Symmetry-Based Representations for Artificial and Biological General Intelligence

Biological intelligence is remarkable in its ability to produce complex ...

Please sign up or login with your details

Forgot password? Click here to reset