Enriching Word Vectors with Subword Information

07/15/2016
by   Piotr Bojanowski, et al.
Facebook
0

Continuous word representations, trained on large unlabeled corpora are useful for many natural language processing tasks. Popular models that learn such representations ignore the morphology of words, by assigning a distinct vector to each word. This is a limitation, especially for languages with large vocabularies and many rare words. In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams. A vector representation is associated to each character n-gram; words being represented as the sum of these representations. Our method is fast, allowing to train models on large corpora quickly and allows us to compute word representations for words that did not appear in the training data. We evaluate our word representations on nine different languages, both on word similarity and analogy tasks. By comparing to recently proposed morphological word representations, we show that our vectors achieve state-of-the-art performance on these tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

07/20/2020

Morphological Skip-Gram: Using morphological knowledge to improve word representation

Natural language processing models have attracted much interest in the d...
02/19/2018

Learning Word Vectors for 157 Languages

Distributed word representations, or word vectors, have recently been ap...
08/18/2015

Probabilistic Modelling of Morphologically Rich Languages

This thesis investigates how the sub-structure of words can be accounted...
04/16/2019

A Systematic Study of Leveraging Subword Information for Learning Word Representations

The use of subword-level information (e.g., characters, character n-gram...
05/19/2023

Contextualized Word Vector-based Methods for Discovering Semantic Differences with No Training nor Word Alignment

In this paper, we propose methods for discovering semantic differences i...
06/10/2016

WordNet2Vec: Corpora Agnostic Word Vectorization Method

A complex nature of big data resources demands new methods for structuri...
12/10/2019

An Ensemble Method for Producing Word Representations for the Greek Language

In this paper we present a new ensemble method, Continuous Bag-of-Skip-g...

Code Repositories

fastText

Library for fast text representation and classification.


view repo

fastText.py

A Python interface for Facebook fastText


view repo

JFastText

Java interface for fastText


view repo

Please sign up or login with your details

Forgot password? Click here to reset