Learning to Embed Sentences Using Attentive Recursive Trees

11/06/2018
by   Jiaxin Shi, et al.
0

Sentence embedding is an effective feature representation for most deep learning-based NLP tasks. One prevailing line of methods is using recursive latent tree-structured networks to embed sentences with task-specific structures. However, existing models have no explicit mechanism to emphasize task-informative words in the tree structure. To this end, we propose an Attentive Recursive Tree model (AR-Tree), where the words are dynamically located according to their importance in the task. Specifically, we construct the latent tree for a sentence in a proposed important-first strategy, and place more attentive words nearer to the root; thus, AR-Tree can inherently emphasize important words during the bottom-up composition of the sentence embedding. We propose an end-to-end reinforced training strategy for AR-Tree, which is demonstrated to consistently outperform, or be at least comparable to, the state-of-the-art sentence embedding methods on three sentence understanding tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/10/2016

Modelling Sentence Pairs with Tree-structured Attentive Encoder

We describe an attentive encoder that combines tree-structured recursive...
research
08/18/2018

Learning to Compose over Tree Structures via POS Tags

Recursive Neural Network (RecNN), a type of models which compose words o...
research
06/25/2022

Construct a Sentence with Multiple Specified Words

This paper demonstrates a task to finetune a BART model so it can constr...
research
11/28/2016

Learning to Compose Words into Sentences with Reinforcement Learning

We use reinforcement learning to learn tree-structured neural networks f...
research
07/20/2023

Efficient Beam Tree Recursion

Beam Tree Recursive Neural Network (BT-RvNN) was recently proposed as a ...
research
05/22/2020

SentPWNet: A Unified Sentence Pair Weighting Network for Task-specific Sentence Embedding

Pair-based metric learning has been widely adopted to learn sentence emb...
research
05/31/2023

Assessing Word Importance Using Models Trained for Semantic Tasks

Many NLP tasks require to automatically identify the most significant wo...

Please sign up or login with your details

Forgot password? Click here to reset