research
∙
07/20/2023
Long-Tail Theory under Gaussian Mixtures
We suggest a simple Gaussian mixture model for data generation that comp...
share
research
∙
04/26/2022
From Hyperbolic Geometry Back to Word Embeddings
We choose random points in the hyperbolic disc and claim that these poin...
research
∙
11/12/2021
Speeding Up Entmax
Softmax is the de facto standard in modern neural networks for language ...
research
∙
03/02/2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics
There is an ongoing debate in the NLP community whether modern language ...
research
∙
02/27/2020
Binarized PMI Matrix: Bridging Word Embeddings and Hyperbolic Spaces
We show analytically that removing sigmoid transformation in the SGNS ob...
research
∙
12/23/2019
Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings
We show that the skip-gram embedding of any word can be decomposed into ...
research
∙
09/30/2019
A Critique of the Smooth Inverse Frequency Sentence Embeddings
We critically review the smooth inverse frequency sentence embedding met...
research
∙
09/21/2019
Low-Rank Approximation of Matrices for PMI-based Word Embeddings
We perform an empirical evaluation of several methods of low-rank approx...
research
∙
02/26/2019
Context Vectors are Reflections of Word Vectors in Half the Dimensions
This paper takes a step towards theoretical analysis of the relationship...
research
∙
02/08/2019
Fourier Neural Networks: A Comparative Study
We review neural network architectures which were motivated by Fourier s...
research
∙
02/23/2018
Reusing Weights in Subword-aware Neural Language Models
We propose several ways of reusing subword embeddings and other weights ...
research
∙
09/02/2017
Patterns versus Characters in Subword-aware Neural Language Modeling
Words in some natural languages can have a composite structure. Elements...
research
∙
07/20/2017