Adversarial Contrastive Estimation

05/09/2018
by   Avishek Bose, et al.
0

Learning by contrasting positive and negative samples is a general strategy adopted by many methods. Noise contrastive estimation (NCE) for word embeddings and translating embeddings for knowledge graphs are examples in NLP employing this approach. In this work, we view contrastive learning as an abstraction of all such methods and augment the negative sampler into a mixture distribution containing an adversarially learned sampler. The resulting adaptive sampler finds harder negative examples, which forces the main model to learn a better representation of the data. We evaluate our proposal on learning word embeddings, order embeddings and knowledge graph embeddings and observe both faster convergence and improved results on multiple metrics.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/06/2022

Improving Contrastive Learning of Sentence Embeddings with Case-Augmented Positives and Retrieved Negatives

Following SimCSE, contrastive learning based methods have achieved the s...
research
04/01/2022

Automatic Biomedical Term Clustering by Learning Fine-grained Term Representations

Term clustering is important in biomedical knowledge graph construction....
research
04/24/2023

Hierarchical Contrastive Learning Enhanced Heterogeneous Graph Neural Network

Heterogeneous graph neural networks (HGNNs) as an emerging technique hav...
research
09/23/2020

Structure Aware Negative Sampling in Knowledge Graphs

Learning low-dimensional representations for entities and relations in k...
research
11/09/2022

Combining Contrastive Learning and Knowledge Graph Embeddings to develop medical word embeddings for the Italian language

Word embeddings play a significant role in today's Natural Language Proc...
research
10/21/2019

Improving Word Representations: A Sub-sampled Unigram Distribution for Negative Sampling

Word2Vec is the most popular model for word representation and has been ...
research
06/03/2022

Contrastive learning unifies t-SNE and UMAP

Neighbor embedding methods t-SNE and UMAP are the de facto standard for ...

Please sign up or login with your details

Forgot password? Click here to reset