Equivariant Transduction through Invariant Alignment

09/22/2022
by   Jennifer C. White, et al.
0

The ability to generalize compositionally is key to understanding the potentially infinite number of sentences that can be constructed in a human language from only a finite number of words. Investigating whether NLP models possess this ability has been a topic of interest: SCAN (Lake and Baroni, 2018) is one task specifically proposed to test for this property. Previous work has achieved impressive empirical results using a group-equivariant neural network that naturally encodes a useful inductive bias for SCAN (Gordon et al., 2020). Inspired by this, we introduce a novel group-equivariant architecture that incorporates a group-invariant hard alignment mechanism. We find that our network's structure allows it to develop stronger equivariance properties than existing group-equivariant approaches. We additionally find that it outperforms previous group-equivariant networks empirically on the SCAN task. Our results suggest that integrating group-equivariance into a variety of neural architectures is a potentially fruitful avenue of research, and demonstrate the value of careful analysis of the theoretical properties of such architectures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/09/2019

Seesaw-Net: Convolution Neural Network With Uneven Group Convolution

In this paper, we are interested in boosting the representation capabili...
research
03/25/2021

Beyond permutation equivariance in graph networks

We introduce a novel architecture for graph networks which is equivarian...
research
09/07/2022

Bispectral Neural Networks

We present a novel machine learning architecture, Bispectral Neural Netw...
research
03/16/2022

Graph Neural Networks for Multiparallel Word Alignment

After a period of decrease, interest in word alignments is increasing ag...
research
05/08/2020

On Designing Secure and Robust Scan Chain for Protecting Obfuscated Logic

In this paper, we assess the security and testability of the state-of-th...
research
05/26/2023

Investigating how ReLU-networks encode symmetries

Many data symmetries can be described in terms of group equivariance and...
research
12/14/2019

Some group-theoretical results on Feistel Networks in a long-key scenario

The study of the trapdoors that can be hidden in a block cipher is and h...

Please sign up or login with your details

Forgot password? Click here to reset