A Functional Perspective on Learning Symmetric Functions with Neural Networks

08/16/2020
by   Aaron Zweig, et al.
13

Symmetric functions, which take as input an unordered, fixed-size set, are known to be universally representable by neural networks that enforce permutation invariance. However, these architectures only give guarantees for fixed input sizes, yet in many practical scenarios, such as particle physics, a relevant notion of generalization should include varying the input size. In this paper, we embed symmetric functions (of any size) as functions over probability measures, and study the ability of neural networks defined over this space of measures to represent and learn in that space. By focusing on shallow architectures, we establish approximation and generalization bounds under different choices of regularization (such as RKHS and variation norms), that capture a hierarchy of functional spaces with increasing amount of non-linear learning. The resulting models can be learnt efficiently and enjoy generalization guarantees that extend across input sizes, as we verify empirically.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/02/2022

Exponential Separations in Symmetric Neural Networks

In this work we demonstrate a novel separation between symmetric neural ...
research
07/01/2019

On Symmetry and Initialization for Neural Networks

This work provides an additional step in the theoretical understanding o...
research
03/22/2023

Anti-symmetric Barron functions and their approximation with sums of determinants

A fundamental problem in quantum physics is to encode functions that are...
research
03/09/2023

On the Expressiveness and Generalization of Hypergraph Neural Networks

This extended abstract describes a framework for analyzing the expressiv...
research
05/19/2023

Complexity of Feed-Forward Neural Networks from the Perspective of Functional Equivalence

In this paper, we investigate the complexity of feed-forward neural netw...
research
02/24/2021

Abelian Neural Networks

We study the problem of modeling a binary operation that satisfies some ...
research
02/14/2023

The Geometry of Neural Nets' Parameter Spaces Under Reparametrization

Model reparametrization – transforming the parameter space via a bijecti...

Please sign up or login with your details

Forgot password? Click here to reset