DeepAI AI Chat
Log In Sign Up

A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Models

by   Usman Naseem, et al.

Word representation has always been an important research area in the history of natural language processing (NLP). Understanding such complex text data is imperative, given that it is rich in information and can be used widely across various applications. In this survey, we explore different word representation models and its power of expression, from the classical to modern-day state-of-the-art word representation language models (LMS). We describe a variety of text representation methods, and model designs have blossomed in the context of NLP, including SOTA LMs. These models can transform large volumes of text into effective vector representations capturing the same semantic information. Further, such representations can be utilized by various machine learning (ML) algorithms for a variety of NLP related tasks. In the end, this survey briefly discusses the commonly used ML and DL based classifiers, evaluation metrics and the applications of these word embeddings in different NLP tasks.


page 1

page 2

page 3

page 4


Meta-Embeddings for Natural Language Inference and Semantic Similarity tasks

Word Representations form the core component for almost all advanced Nat...

Subword ELMo

Embedding from Language Models (ELMo) has shown to be effective for impr...

Extending the Abstraction of Personality Types based on MBTI with Machine Learning and Natural Language Processing

A data-centric approach with Natural Language Processing (NLP) to predic...

RETRO: Relation Retrofitting For In-Database Machine Learning on Textual Data

There are massive amounts of textual data residing in databases, valuabl...

How to Evaluate Word Representations of Informal Domain?

Diverse word representations have surged in most state-of-the-art natura...

SECRET: Semantically Enhanced Classification of Real-world Tasks

Supervised machine learning (ML) algorithms are aimed at maximizing clas...

Automatic coding of students' writing via Contrastive Representation Learning in the Wasserstein space

Qualitative analysis of verbal data is of central importance in the lear...