A Cookbook of Self-Supervised Learning

04/24/2023

∙

Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning. Yet, much like cooking, training SSL methods is a delicate art with a high barrier to entry. While many components are familiar, successfully training a SSL method involves a dizzying set of choices from the pretext tasks to training hyper-parameters. Our goal is to lower the barrier to entry into SSL research by laying the foundations and latest SSL recipes in the style of a cookbook. We hope to empower the curious researcher to navigate the terrain of methods, understand the role of the various knobs, and gain the know-how required to explore how delicious SSL can be.

READ FULL TEXT

A Cookbook of Self-Supervised Learning

Homomorphic Self-Supervised Learning

An Empirical Study of Training Self-Supervised Vision Transformers

Accelerating Self-Supervised Learning via Efficient Training Strategies

Re-entry Prediction for Online Conversations via Self-Supervised Learning

Self-supervised Feature Enhancement: Applying Internal Pretext Task to Supervised Learning

Blockwise Self-Supervised Learning at Scale

Self-Supervised Vessel Enhancement Using Flow-Based Consistencies

A Cookbook of Self-Supervised Learning

Related Research

Homomorphic Self-Supervised Learning

An Empirical Study of Training Self-Supervised Vision Transformers

Accelerating Self-Supervised Learning via Efficient Training Strategies

Re-entry Prediction for Online Conversations via Self-Supervised Learning

Self-supervised Feature Enhancement: Applying Internal Pretext Task to Supervised Learning

Blockwise Self-Supervised Learning at Scale

Self-Supervised Vessel Enhancement Using Flow-Based Consistencies