DeepAI AI Chat
Log In Sign Up

Representation learning for very short texts using weighted word embedding aggregation

07/02/2016
by   Cedric De Boom, et al.
Ghent University
0

Short text messages such as tweets are very noisy and sparse in their use of vocabulary. Traditional textual representations, such as tf-idf, have difficulty grasping the semantic meaning of such texts, which is important in applications such as event detection, opinion mining, news recommendation, etc. We constructed a method based on semantic word embeddings and frequency information to arrive at low-dimensional representations for short texts designed to capture semantic similarity. For this purpose we designed a weight-based model and a learning procedure based on a novel median-based loss function. This paper discusses the details of our model and the optimization methods, together with the experimental results on both Wikipedia and Twitter data. We find that our method outperforms the baseline approaches in the experiments, and that it generalizes well on different word embeddings without retraining. Our method is therefore capable of retaining most of the semantic information in the text, and is applicable out-of-the-box.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/14/2017

Data Sets: Word Embeddings Learned from Tweets and General Data

A word embedding is a low-dimensional, dense and real- valued vector rep...
12/02/2015

Learning Semantic Similarity for Very Short Texts

Levering data on social media, such as Twitter and Facebook, requires in...
10/17/2017

Unsupervised Sentence Representations as Word Information Series: Revisiting TF--IDF

Sentence representation at the semantic level is a challenging task for ...
10/07/2020

MuSeM: Detecting Incongruent News Headlines using Mutual Attentive Semantic Matching

Measuring the congruence between two texts has several useful applicatio...
05/21/2017

Learning Semantic Relatedness From Human Feedback Using Metric Learning

Assessing the degree of semantic relatedness between words is an importa...
09/22/2022

Homophone Reveals the Truth: A Reality Check for Speech2Vec

Generating spoken word embeddings that possess semantic information is a...
11/15/2022

Type Information Utilized Event Detection via Multi-Channel GNNs in Electrical Power Systems

Event detection in power systems aims to identify triggers and event typ...

Code Repositories

wordKernel

An approach to word vectorization that represents words by their weighted proximity to other words in corpus


view repo