DeepAI AI Chat
Log In Sign Up

Learning Features from Co-occurrences: A Theoretical Analysis

by   Yanpeng Li, et al.

Representing a word by its co-occurrences with other words in context is an effective way to capture the meaning of the word. However, the theory behind remains a challenge. In this work, taking the example of a word classification task, we give a theoretical analysis of the approaches that represent a word X by a function f(P(C|X)), where C is a context feature, P(C|X) is the conditional probability estimated from a text corpus, and the function f maps the co-occurrence measure to a prediction score. We investigate the impact of context feature C and the function f. We also explain the reasons why using the co-occurrences with multiple context features may be better than just using a single one. In addition, some of the results shed light on the theory of feature learning and machine learning in general.


page 1

page 2

page 3

page 4


Context based Analysis of Lexical Semantics for Hindi Language

A word having multiple senses in a text introduces the lexical semantic ...

Context-theoretic Semantics for Natural Language: an Algebraic Framework

Techniques in which words are represented as vectors have proved useful ...

A Context-theoretic Framework for Compositionality in Distributional Semantics

Techniques in which words are represented as vectors have proved useful ...

A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

Motivations like domain adaptation, transfer learning, and feature learn...

Contextual Skipgram: Training Word Representation Using Context Information

The skip-gram (SG) model learns word representation by predicting the wo...

A Classification Approach to Word Prediction

The eventual goal of a language model is to accurately predict the value...

The Satisfiability of Extended Word Equations: The Boundary Between Decidability and Undecidability

The study of word equations (or the existential theory of equations over...