A Survey on Word Meta-Embedding Learning

04/25/2022
by   Danushka Bollegala, et al.
0

Meta-embedding (ME) learning is an emerging approach that attempts to learn more accurate word embeddings given existing (source) word embeddings as the sole input. Due to their ability to incorporate semantics from multiple source embeddings in a compact manner with superior performance, ME learning has gained popularity among practitioners in NLP. To the best of our knowledge, there exist no prior systematic survey on ME learning and this paper attempts to fill this need. We classify ME learning methods according to multiple factors such as whether they (a) operate on static or contextualised embeddings, (b) trained in an unsupervised manner or (c) fine-tuned for a particular task/domain. Moreover, we discuss the limitations of existing ME learning methods and highlight potential future research directions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/26/2022

Learning Meta Word Embeddings by Unsupervised Weighted Concatenation of Source Embeddings

Given multiple source word embeddings learnt using diverse algorithms an...
research
09/19/2017

Think Globally, Embed Locally --- Locally Linear Meta-embedding of Words

Distributed word embeddings have shown superior performances in numerous...
research
11/05/2020

Learning Efficient Task-Specific Meta-Embeddings with Word Prisms

Word embeddings are trained to predict word cooccurrence statistics, whi...
research
04/14/2018

Frustratingly Easy Meta-Embedding -- Computing Meta-Embeddings by Averaging Source Word Embeddings

Creating accurate meta-embeddings from pre-trained source embeddings has...
research
05/19/2022

Gender Bias in Meta-Embeddings

Combining multiple source embeddings to create meta-embeddings is consid...
research
09/12/2018

Graph Convolutional Networks based Word Embeddings

Recently, word embeddings have been widely adopted across several NLP ap...
research
09/29/2019

Meta-Graph Based HIN Spectral Embedding: Methods, Analyses, and Insights

In this work, we propose to study the utility of different meta-graphs, ...

Please sign up or login with your details

Forgot password? Click here to reset