Extrapolation in NLP

05/17/2018
by   Jeff Mitchell, et al.
0

We argue that extrapolation to examples outside the training space will often be easier for models that capture global structures, rather than just maximise their local fit to the training data. We show that this is true for two popular models: the Decomposable Attention Model and word2vec.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/09/2022

Where's the Learning in Representation Learning for Compositional Semantics and the Case of Thematic Fit

Observing that for certain NLP tasks, such as semantic role prediction o...
research
07/02/2023

Partial-label Learning with Mixed Closed-set and Open-set Out-of-candidate Examples

Partial-label learning (PLL) relies on a key assumption that the true la...
research
05/21/2021

Sheaves as a Framework for Understanding and Interpreting Model Fit

As data grows in size and complexity, finding frameworks which aid in in...
research
06/27/2023

Large Language Models as Annotators: Enhancing Generalization of NLP Models at Minimal Cost

State-of-the-art supervised NLP models achieve high accuracy but are als...
research
03/01/2019

Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers

When considering sparse motion capture marker data, one typically strugg...
research
07/23/2022

A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictors

In this work we establish an algorithm and distribution independent non-...

Please sign up or login with your details

Forgot password? Click here to reset