DeepAI AI Chat
Log In Sign Up

Embedding Transfer with Label Relaxation for Improved Metric Learning

by   Sungyeon Kim, et al.

This paper presents a novel method for embedding transfer, a task of transferring knowledge of a learned embedding model to another. Our method exploits pairwise similarities between samples in the source embedding space as the knowledge, and transfers them through a loss used for learning target embedding models. To this end, we design a new loss called relaxed contrastive loss, which employs the pairwise similarities as relaxed labels for inter-sample relations. Our loss provides a rich supervisory signal beyond class equivalence, enables more important pairs to contribute more to training, and imposes no restriction on manifolds of target embedding spaces. Experiments on metric learning benchmarks demonstrate that our method largely improves performance, or reduces sizes and output dimensions of target models effectively. We further show that it can be also used to enhance quality of self-supervised representation and performance of classification models. In all the experiments, our method clearly outperforms existing embedding transfer techniques.


page 2

page 5

page 7

page 11

page 12

page 14

page 15


Self-Taught Metric Learning without Labels

We present a novel self-taught framework for unsupervised metric learnin...

Learning to Generate Novel Classes for Deep Metric Learning

Deep metric learning aims to learn an embedding space where the distance...

Deep Metric Learning with Spherical Embedding

Deep metric learning has attracted much attention in recent years, due t...

Diversified Mutual Learning for Deep Metric Learning

Mutual learning is an ensemble training strategy to improve generalizati...

Deep Metric Learning with Density Adaptivity

The problem of distance metric learning is mostly considered from the pe...

Integrating Language Guidance into Vision-based Deep Metric Learning

Deep Metric Learning (DML) proposes to learn metric spaces which encode ...

Secure Metric Learning via Differential Pairwise Privacy

Distance Metric Learning (DML) has drawn much attention over the last tw...