-
Self-supervised Learning: Generative or Contrastive
Deep supervised learning has achieved great success in the last decade. ...
read it
-
A Survey on Contrastive Self-supervised Learning
Self-supervised learning has gained popularity because of its ability to...
read it
-
How to represent part-whole hierarchies in a neural network
This paper does not describe a working system. Instead, it presents a si...
read it
-
A Primer on Contrastive Pretraining in Language Processing: Methods, Lessons Learned and Perspectives
Modern natural language processing (NLP) methods employ self-supervised ...
read it
-
A Mutual Information Maximization Perspective of Language Representation Learning
We show state-of-the-art word representation learning methods maximize a...
read it
-
CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning
More than half of the 7,000 languages in the world are in imminent dange...
read it
-
A Review on Deep Learning Techniques for Video Prediction
The ability to predict, anticipate and reason about future outcomes is a...
read it
Contrastive Representation Learning: A Framework and Review
Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.
READ FULL TEXT
Comments
There are no comments yet.