DeepAI AI Chat
Log In Sign Up

Embedding Transfer with Label Relaxation for Improved Metric Learning

03/27/2021
by   Sungyeon Kim, et al.
0

This paper presents a novel method for embedding transfer, a task of transferring knowledge of a learned embedding model to another. Our method exploits pairwise similarities between samples in the source embedding space as the knowledge, and transfers them through a loss used for learning target embedding models. To this end, we design a new loss called relaxed contrastive loss, which employs the pairwise similarities as relaxed labels for inter-sample relations. Our loss provides a rich supervisory signal beyond class equivalence, enables more important pairs to contribute more to training, and imposes no restriction on manifolds of target embedding spaces. Experiments on metric learning benchmarks demonstrate that our method largely improves performance, or reduces sizes and output dimensions of target models effectively. We further show that it can be also used to enhance quality of self-supervised representation and performance of classification models. In all the experiments, our method clearly outperforms existing embedding transfer techniques.

READ FULL TEXT

page 2

page 5

page 7

page 11

page 12

page 14

page 15

05/04/2022

Self-Taught Metric Learning without Labels

We present a novel self-taught framework for unsupervised metric learnin...
01/04/2022

Learning to Generate Novel Classes for Deep Metric Learning

Deep metric learning aims to learn an embedding space where the distance...
11/05/2020

Deep Metric Learning with Spherical Embedding

Deep metric learning has attracted much attention in recent years, due t...
09/09/2020

Diversified Mutual Learning for Deep Metric Learning

Mutual learning is an ensemble training strategy to improve generalizati...
09/09/2019

Deep Metric Learning with Density Adaptivity

The problem of distance metric learning is mostly considered from the pe...
03/16/2022

Integrating Language Guidance into Vision-based Deep Metric Learning

Deep Metric Learning (DML) proposes to learn metric spaces which encode ...
03/30/2020

Secure Metric Learning via Differential Pairwise Privacy

Distance Metric Learning (DML) has drawn much attention over the last tw...