Sentence transition matrix: An efficient approach that preserves sentence semantics

01/16/2019
by   Myeongjun Jang, et al.
0

Sentence embedding is a significant research topic in the field of natural language processing (NLP). Generating sentence embedding vectors reflecting the intrinsic meaning of a sentence is a key factor to achieve an enhanced performance in various NLP tasks such as sentence classification and document summarization. Therefore, various sentence embedding models based on supervised and unsupervised learning have been proposed after the advent of researches regarding the distributed representation of words. They were evaluated through semantic textual similarity (STS) tasks, which measure the degree of semantic preservation of a sentence and neural network-based supervised embedding models generally yielded state-of-the-art performance. However, these models have a limitation in that they have multiple parameters to update, thereby requiring a tremendous amount of labeled training data. In this study, we propose an efficient approach that learns a transition matrix that refines a sentence embedding vector to reflect the latent semantic meaning of a sentence. The proposed method has two practical advantages; (1) it can be applied to any sentence embedding method, and (2) it can achieve robust performance in STS tasks irrespective of the number of training examples.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/16/2018

Paraphrase Thought: Sentence Embedding Module Imitating Human Language Recognition

Sentence embedding is an important research topic in natural language pr...
research
06/04/2023

Sen2Pro: A Probabilistic Perspective to Sentence Embedding from Pre-trained Language Model

Sentence embedding is one of the most fundamental tasks in Natural Langu...
research
09/07/2021

PAUSE: Positive and Annealed Unlabeled Sentence Embedding

Sentence embedding refers to a set of effective and versatile techniques...
research
02/21/2023

NLPLego: Assembling Test Generation for Natural Language Processing Applications

The development of modern NLP applications often relies on various bench...
research
08/08/2023

A Comparative Study of Sentence Embedding Models for Assessing Semantic Variation

Analyzing the pattern of semantic variation in long real-world texts suc...
research
04/05/2021

WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach

Producing the embedding of a sentence in an unsupervised way is valuable...
research
02/22/2020

Efficient Sentence Embedding via Semantic Subspace Analysis

A novel sentence embedding method built upon semantic subspace analysis,...

Please sign up or login with your details

Forgot password? Click here to reset