Short Text Clustering with Transformers

01/31/2021
by   Leonid Pugachev, et al.
0

Recent techniques for the task of short text clustering often rely on word embeddings as a transfer learning component. This paper shows that sentence vector representations from Transformers in conjunction with different clustering methods can be successfully applied to address the task. Furthermore, we demonstrate that the algorithm of enhancement of clustering via iterative classification can further improve initial clustering performance with different classifiers, including those based on pre-trained Transformer language models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/31/2020

Enhancement of Short Text Clustering by Iterative Classification

Short text clustering is a challenging task due to the lack of signal co...
research
04/20/2023

CEIL: A General Classification-Enhanced Iterative Learning Framework for Text Clustering

Text clustering, as one of the most fundamental challenges in unsupervis...
research
09/06/2023

Combining pre-trained Vision Transformers and CIDER for Out Of Domain Detection

Out-of-domain (OOD) detection is a crucial component in industrial appli...
research
09/05/2017

Language Modeling by Clustering with Word Embeddings for Text Readability Assessment

We present a clustering-based language model using word embeddings for t...
research
01/01/2017

Self-Taught Convolutional Neural Networks for Short Text Clustering

Short text clustering is a challenging problem due to its sparseness of ...
research
01/22/2021

Multilingual Pre-Trained Transformers and Convolutional NN Classification Models for Technical Domain Identification

In this paper, we present a transfer learning system to perform technica...
research
05/29/2023

Transformer Language Models Handle Word Frequency in Prediction Head

Prediction head is a crucial component of Transformer language models. D...

Please sign up or login with your details

Forgot password? Click here to reset