InfoDCL: A Distantly Supervised Contrastive Learning Framework for Social Meaning

03/15/2022
by   Chiyu Zhang, et al.
0

Existing supervised contrastive learning frameworks suffer from two major drawbacks: (i) they depend on labeled data, which is limited for the majority of tasks in real-world, and (ii) they incorporate inter-class relationships based on instance-level information, while ignoring corpus-level information, for weighting negative samples. To mitigate these challenges, we propose an effective distantly supervised contrastive learning framework (InfoDCL) that makes use of naturally occurring surrogate labels in the context of contrastive learning and employs pointwise mutual information to leverage corpus-level information. Our framework outperforms an extensive set of existing contrastive learning methods (self-supervised, supervised, and weakly supervised) on a wide range of social meaning tasks (in-domain and out-of-domain), in both the general and few-shot settings. Our method is also language-agnostic, as we demonstrate on three languages in addition to English.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/28/2023

Multi-network Contrastive Learning Based on Global and Local Representations

The popularity of self-supervised learning has made it possible to train...
research
10/05/2020

EqCo: Equivalent Rules for Self-supervised Contrastive Learning

In this paper, we propose a method, named EqCo (Equivalent Rules for Con...
research
09/06/2022

Fun2Vec:a Contrastive Learning Framework of Function-level Representation for Binary

Function-level binary code similarity detection is essential in the fiel...
research
07/10/2023

Hate Speech Detection via Dual Contrastive Learning

The fast spread of hate speech on social media impacts the Internet envi...
research
10/04/2022

COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation

Annotation of large-scale 3D data is notoriously cumbersome and costly. ...
research
10/15/2021

Surrogate- and invariance-boosted contrastive learning for data-scarce applications in science

Deep learning techniques have been increasingly applied to the natural s...

Please sign up or login with your details

Forgot password? Click here to reset