Log In Sign Up

Should Semantic Vector Composition be Explicit? Can it be Linear

by   Dominic Widdows, et al.

Vector representations have become a central element in semantic language modelling, leading to mathematical overlaps with many fields including quantum theory. Compositionality is a core goal for such representations: given representations for `wet' and `fish', how should the concept `wet fish' be represented? This position paper surveys this question from two points of view. The first considers the question of whether an explicit mathematical representation can be successful using only tools from within linear algebra, or whether other mathematical tools are needed. The second considers whether semantic vector composition should be explicitly described mathematically, or whether it can be a model-internal side-effect of training a neural network. This paper is intended as a survey and motivation for discussion, and does not claim to give definitive answers to the questions posed. We speculate that these questions are related, and that the nonlinear operators used in implicitly compositional language models may inform explicit compositional modelling.


page 1

page 2

page 3

page 4


Neural Compositional Denotational Semantics for Question Answering

Answering compositional questions requiring multi-step reasoning is chal...

Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models

Compositionality, the phenomenon where the meaning of a phrase can be de...

Categories of Semantic Concepts

Modelling concept representation is a foundational problem in the study ...

Compositional Distributional Semantics with Compact Closed Categories and Frobenius Algebras

This thesis contributes to ongoing research related to the categorical c...

Assessing Composition in Sentence Vector Representations

An important component of achieving language understanding is mastering ...

Compositional Model Repositories via Dynamic Constraint Satisfaction with Order-of-Magnitude Preferences

The predominant knowledge-based approach to automated model construction...