Semantic Granularity Metric Learning for Visual Search

11/14/2019
by   Dipu Manandhar, et al.
36

Deep metric learning applied to various applications has shown promising results in identification, retrieval and recognition. Existing methods often do not consider different granularity in visual similarity. However, in many domain applications, images exhibit similarity at multiple granularities with visual semantic concepts, e.g. fashion demonstrates similarity ranging from clothing of the exact same instance to similar looks/design or a common category. Therefore, training image triplets/pairs used for metric learning inherently possess different degree of information. However, the existing methods often treats them with equal importance during training. This hinders capturing the underlying granularities in feature similarity required for effective visual search. In view of this, we propose a new deep semantic granularity metric learning (SGML) that develops a novel idea of leveraging attribute semantic space to capture different granularity of similarity, and then integrate this information into deep metric learning. The proposed method simultaneously learns image attributes and embeddings using multitask CNNs. The two tasks are not only jointly optimized but are further linked by the semantic granularity similarity mappings to leverage the correlations between the tasks. To this end, we propose a new soft-binomial deviance loss that effectively integrates the degree of information in training samples, which helps to capture visual similarity at multiple granularities. Compared to recent ensemble-based methods, our framework is conceptually elegant, computationally simple and provides better performance. We perform extensive experiments on benchmark metric learning datasets and demonstrate that our method outperforms recent state-of-the-art methods, e.g., 1-4.5% improvement in Recall@1 over the previous state-of-the-arts [1],[2] on DeepFashion In-Shop dataset.

READ FULL TEXT

page 1

page 2

page 9

research
04/04/2019

Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning

Deep metric learning, which learns discriminative features to process im...
research
09/27/2019

Visual Explanation for Deep Metric Learning

This work explores the visual explanation for deep metric learning and i...
research
10/07/2022

Learning to embed semantic similarity for joint image-text retrieval

We present a deep learning approach for learning the joint semantic embe...
research
04/28/2020

DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning

Visual Similarity plays an important role in many computer vision applic...
research
04/28/2020

DiVA: Diverse Visual Feature Aggregation forDeep Metric Learning

Visual Similarity plays an important role in many computer vision applic...
research
09/20/2019

Deep Metric Learning using Similarities from Nonlinear Rank Approximations

In recent years, deep metric learning has achieved promising results in ...
research
03/22/2021

Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales

This paper introduces a new fundamental characteristic, , the dynamic ra...

Please sign up or login with your details

Forgot password? Click here to reset