Semantic Granularity Metric Learning for Visual Search

by   Dipu Manandhar, et al.

Deep metric learning applied to various applications has shown promising results in identification, retrieval and recognition. Existing methods often do not consider different granularity in visual similarity. However, in many domain applications, images exhibit similarity at multiple granularities with visual semantic concepts, e.g. fashion demonstrates similarity ranging from clothing of the exact same instance to similar looks/design or a common category. Therefore, training image triplets/pairs used for metric learning inherently possess different degree of information. However, the existing methods often treats them with equal importance during training. This hinders capturing the underlying granularities in feature similarity required for effective visual search. In view of this, we propose a new deep semantic granularity metric learning (SGML) that develops a novel idea of leveraging attribute semantic space to capture different granularity of similarity, and then integrate this information into deep metric learning. The proposed method simultaneously learns image attributes and embeddings using multitask CNNs. The two tasks are not only jointly optimized but are further linked by the semantic granularity similarity mappings to leverage the correlations between the tasks. To this end, we propose a new soft-binomial deviance loss that effectively integrates the degree of information in training samples, which helps to capture visual similarity at multiple granularities. Compared to recent ensemble-based methods, our framework is conceptually elegant, computationally simple and provides better performance. We perform extensive experiments on benchmark metric learning datasets and demonstrate that our method outperforms recent state-of-the-art methods, e.g., 1-4.5% improvement in Recall@1 over the previous state-of-the-arts [1],[2] on DeepFashion In-Shop dataset.


page 1

page 2

page 9


Signal-to-Noise Ratio: A Robust Distance Metric for Deep Metric Learning

Deep metric learning, which learns discriminative features to process im...

Visual Explanation for Deep Metric Learning

This work explores the visual explanation for deep metric learning and i...

Deep Metric Learning Beyond Binary Supervision

Metric Learning for visual similarity has mostly adopted binary supervis...

DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning

Visual Similarity plays an important role in many computer vision applic...

DiVA: Diverse Visual Feature Aggregation forDeep Metric Learning

Visual Similarity plays an important role in many computer vision applic...

Deep Metric Learning using Similarities from Nonlinear Rank Approximations

In recent years, deep metric learning has achieved promising results in ...

Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales

This paper introduces a new fundamental characteristic, , the dynamic ra...