A Framework for Measuring Compositional Inductive Bias

03/06/2021 ∙ by Hugh Perkins, et al. ∙ 0

We present a framework for measuring the compositional inductive bias of models in the context of emergent communications. We devise corrupted compositional grammars that probe for limitations in the compositional inductive bias of frequently used models. We use these corrupted compositional grammars to compare and contrast a wide range of models, and to compare the choice of soft, Gumbel, and discrete representations. We propose a hierarchical model which might show an inductive bias towards relocatable atomic groups of tokens, thus potentially encouraging the emergence of words. We experiment with probing for the compositional inductive bias of sender and receiver networks in isolation, and also placed end-to-end, as an auto-encoder.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

Code Repositories

compositional-inductive-bias

Code for paper "A Framework for Measuring Compositional Inductive Bias", Hugh Perkins, 2021 (https://arxiv.org/abs/2103.04180)


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.