Analysis of Bag-of-n-grams Representation's Properties Based on Textual Reconstruction

09/18/2018
by   Qi Huang, et al.
0

Despite its simplicity, bag-of-n-grams sen- tence representation has been found to excel in some NLP tasks. However, it has not re- ceived much attention in recent years and fur- ther analysis on its properties is necessary. We propose a framework to investigate the amount and type of information captured in a general- purposed bag-of-n-grams sentence represen- tation. We first use sentence reconstruction as a tool to obtain bag-of-n-grams representa- tion that contains general information of the sentence. We then run prediction tasks (sen- tence length, word content, phrase content and word order) using the obtained representation to look into the specific type of information captured in the representation. Our analysis demonstrates that bag-of-n-grams representa- tion does contain sentence structure level in- formation. However, incorporating n-grams with higher order n empirically helps little with encoding more information in general, except for phrase content information.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/06/2023

Static Fuzzy Bag-of-Words: a lightweight sentence embedding algorithm

The introduction of embedding techniques has pushed forward significantl...
research
08/15/2016

Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks

There is a lot of research interest in encoding variable length sentence...
research
09/10/2021

Studying word order through iterative shuffling

As neural language models approach human performance on NLP benchmark ta...
research
11/07/2019

Query-bag Matching with Mutual Coverage for Information-seeking Conversations in E-commerce

Information-seeking conversation system aims at satisfying the informati...
research
08/18/2018

Learning to Compose over Tree Structures via POS Tags

Recursive Neural Network (RecNN), a type of models which compose words o...
research
01/09/2017

Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching

This work studies comparatively two typical sentence matching tasks: tex...

Please sign up or login with your details

Forgot password? Click here to reset