Learning Aggregation Functions

12/15/2020
by   Giovanni Pellegrini, et al.
0

Learning on sets is increasingly gaining attention in the machine learning community, due to its widespread applicability. Typically, representations over sets are computed by using fixed aggregation functions such as sum or maximum. However, recent results showed that universal function representation by sum- (or max-) decomposition requires either highly discontinuous (and thus poorly learnable) mappings, or a latent dimension equal to the maximum number of elements in the set. To mitigate this problem, we introduce LAF (Learning Aggregation Functions), a learnable aggregator for sets of arbitrary cardinality. LAF can approximate several extensively used aggregators (such as average, sum, maximum) as well as more complex functions (e.g. variance and skewness). We report experiments on semi-synthetic and real data showing that LAF outperforms state-of-the-art sum- (max-) decomposition architectures such as DeepSets and library-based architectures like Principal Neighborhood Aggregation.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/18/2019

On Deep Set Learning and the Choice of Aggregations

Recently, it has been shown that many functions on sets can be represent...
01/25/2019

On the Limitations of Representing Functions on Sets

Recent work on the representation of functions on sets has considered th...
02/25/2022

Equilibrium Aggregation: Encoding Sets via Optimization

Processing sets or other unordered, potentially variable-sized inputs in...
02/18/2022

Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via Nonlinear Neighborhood Aggregators

Graph neural networks (GNNs) have achieved great success in many graph l...
06/17/2020

Generalising Recursive Neural Models by Tensor Decomposition

Most machine learning models for structured data encode the structural k...
06/30/2022

Learning Functions on Multiple Sets using Multi-Set Transformers

We propose a general deep architecture for learning functions on multipl...
08/27/2013

Improving Sparse Associative Memories by Escaping from Bogus Fixed Points

The Gripon-Berrou neural network (GBNN) is a recently invented recurrent...