On Universal Equivariant Set Networks

by   Nimrod Segol, et al.

Using deep neural networks that are either invariant or equivariant to permutations in order to learn functions on unordered sets has become prevalent. The most popular, basic models are DeepSets [Zaheer et al. 2017] and PointNet [Qi et al. 2017]. While known to be universal for approximating invariant functions, DeepSets and PointNet are not known to be universal when approximating equivariant set functions. On the other hand, several recent equivariant set architectures have been proven equivariant universal [Sannai et al. 2019], [Keriven et al. 2019], however these models either use layers that are not permutation equivariant (in the standard sense) and/or use higher order tensor variables which are less practical. There is, therefore, a gap in understanding the universality of popular equivariant set models versus theoretical ones. In this paper we close this gap by proving that: (i) PointNet is not equivariant universal; and (ii) adding a single linear transmission layer makes PointNet universal. We call this architecture PointNetST and argue it is the simplest permutation equivariant universal model known to date. Another consequence is that DeepSets is universal, and also PointNetSeg, a popular point cloud segmentation network (used eg, in [Qi et al. 2017]) is universal. The key theoretical tool used to prove the above results is an explicit characterization of all permutation equivariant polynomial layers. Lastly, we provide numerical experiments validating the theoretical results and comparing different permutation equivariant models.


page 1

page 2

page 3

page 5

page 6

page 7

page 8

page 9


Universal approximations of permutation invariant/equivariant functions by deep neural networks

In this paper,we develop a theory of the relationship between permutatio...

Universal Invariant and Equivariant Graph Neural Networks

Graph Neural Networks (GNN) come in many flavors, but should always be e...

Learning Set-equivariant Functions with SWARM Mappings

In this work we propose a new neural network architecture that efficient...

A Unified Taxonomy of Deep Syntactic Relations

This paper analyzes multiple deep-syntactic frameworks with the goal of ...

O(N^2) Universal Antisymmetry in Fermionic Neural Networks

Fermionic neural network (FermiNet) is a recently proposed wavefunction ...

Investigating how ReLU-networks encode symmetries

Many data symmetries can be described in terms of group equivariance and...

Invariant and Equivariant Graph Networks

Invariant and equivariant networks have been successfully used for learn...

Please sign up or login with your details

Forgot password? Click here to reset