Expressiveness and Learnability: A Unifying View for Evaluating Self-Supervised Learning

06/02/2022
by   Yuchen Lu, et al.
0

We propose a unifying view to analyze the representation quality of self-supervised learning (SSL) models without access to supervised labels, while being agnostic to the architecture, learning algorithm or data manipulation used during training. We argue that representations can be evaluated through the lens of expressiveness and learnability. We propose to use the Intrinsic Dimension (ID) to assess expressiveness and introduce Cluster Learnability (CL) to assess learnability. CL is measured as the learning speed of a KNN classifier trained to predict labels obtained by clustering the representations with K-means. We thus combine CL and ID into a single predictor: CLID. Through a large-scale empirical study with a diverse family of SSL algorithms, we find that CLID better correlates with in-distribution model performance than other competing recent evaluation schemes. We also benchmark CLID on out-of-domain generalization, where CLID serves as a predictor of the transfer performance of SSL models on several classification tasks, yielding improvements with respect to the competing baselines.

READ FULL TEXT
research
10/28/2021

Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction

Self-Supervised Learning (SSL) has been shown to learn useful and inform...
research
03/27/2022

How Severe is Benchmark-Sensitivity in Video Self-Supervised Learning?

Despite the recent success of video self-supervised learning, there is m...
research
09/07/2023

Adapting Self-Supervised Representations to Multi-Domain Setups

Current state-of-the-art self-supervised approaches, are effective when ...
research
10/19/2021

Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles

Pretraining convolutional neural networks via self-supervision, and appl...
research
02/18/2022

Masked prediction tasks: a parameter identifiability view

The vast majority of work in self-supervised learning, both theoretical ...
research
02/07/2022

Simple Control Baselines for Evaluating Transfer Learning

Transfer learning has witnessed remarkable progress in recent years, for...
research
11/12/2020

Discriminative, Generative and Self-Supervised Approaches for Target-Agnostic Learning

Supervised learning, characterized by both discriminative and generative...

Please sign up or login with your details

Forgot password? Click here to reset