Structured Embedding Models for Grouped Data

09/28/2017
by   Maja Rudolph, et al.
0

Word embeddings are a powerful approach for analyzing language, and exponential family embeddings (EFE) extend them to other types of data. Here we develop structured exponential family embeddings (S-EFE), a method for discovering embeddings that vary across related groups of data. We study how the word usage of U.S. Congressional speeches varies across states and party affiliation, how words are used differently across sections of the ArXiv, and how the co-purchase patterns of groceries can vary across seasons. Key to the success of our method is that the groups share statistical information. We develop two sharing strategies: hierarchical modeling and amortization. We demonstrate the benefits of this approach in empirical studies of speeches, abstracts, and shopping baskets. We show how S-EFE enables group-specific interpretation of word usage, and outperforms EFE in predicting held-out data.

READ FULL TEXT
research
03/23/2017

Dynamic Bernoulli Embeddings for Language Evolution

Word embeddings are a powerful approach for unsupervised analysis of lan...
research
08/02/2016

Exponential Family Embeddings

Word embeddings are a powerful approach for capturing semantic similarit...
research
10/10/2020

Automated Concatenation of Embeddings for Structured Prediction

Pretrained contextualized embeddings are powerful word representations f...
research
11/11/2020

Exploring the Value of Personalized Word Embeddings

In this paper, we introduce personalized word embeddings, and examine th...
research
03/31/2022

A bilingual approach to specialised adjectives through word embeddings in the karstology domain

We present an experiment in extracting adjectives which express a specif...
research
01/17/2023

Word Embeddings as Statistical Estimators

Word embeddings are a fundamental tool in natural language processing. C...
research
12/19/2022

Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications

Scholarly text is often laden with jargon, or specialized language that ...

Please sign up or login with your details

Forgot password? Click here to reset