Representing ELMo embeddings as two-dimensional text online

03/30/2021
by   Andrey Kutuzov, et al.
0

We describe a new addition to the WebVectors toolkit which is used to serve word embedding models over the Web. The new ELMoViz module adds support for contextualized embedding architectures, in particular for ELMo models. The provided visualizations follow the metaphor of `two-dimensional text' by showing lexical substitutes: words which are most semantically similar in context to the words of the input sentence. The system allows the user to change the ELMo layers from which token embeddings are inferred. It also conveys corpus information about the query words and their lexical substitutes (namely their frequency tiers and parts of speech). The module is well integrated into the rest of the WebVectors toolkit, providing lexical hyperlinks to word representations in static embedding models. Two web services have already implemented the new functionality with pre-trained ELMo models for Russian, Norwegian and English.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/09/2017

Learning to Embed Words in Context for Syntactic Tasks

We present models for embedding words in the context of surrounding word...
research
11/07/2019

Using Dynamic Embeddings to Improve Static Embeddings

How to build high-quality word embeddings is a fundamental research ques...
research
11/29/2017

Embedding Words as Distributions with a Bayesian Skip-gram Model

We introduce a method for embedding words as probability densities in a ...
research
05/31/2020

A Unified Feature Representation for Lexical Connotations

Ideological attitudes and stance are often expressed through subtle mean...
research
06/12/2019

Putting words in context: LSTM language models and lexical ambiguity

In neural network models of language, words are commonly represented usi...
research
11/15/2015

Word Embedding based Correlation Model for Question/Answer Matching

With the development of community based question answering (Q&A) service...
research
02/27/2019

Still a Pain in the Neck: Evaluating Text Representations on Lexical Composition

Building meaningful phrase representations is challenging because phrase...

Please sign up or login with your details

Forgot password? Click here to reset