Towards Interpretable Deep Metric Learning with Structural Matching

08/12/2021
by   Wenliang Zhao, et al.
52

How do the neural networks distinguish two images? It is of critical importance to understand the matching mechanism of deep models for developing reliable intelligent systems for many risky visual applications such as surveillance and access control. However, most existing deep metric learning methods match the images by comparing feature vectors, which ignores the spatial structure of images and thus lacks interpretability. In this paper, we present a deep interpretable metric learning (DIML) method for more transparent embedding learning. Unlike conventional metric learning methods based on feature vector comparison, we propose a structural matching strategy that explicitly aligns the spatial embeddings by computing an optimal matching flow between feature maps of the two images. Our method enables deep models to learn metrics in a more human-friendly way, where the similarity of two images can be decomposed to several part-wise similarities and their contributions to the overall similarity. Our method is model-agnostic, which can be applied to off-the-shelf backbone networks and metric learning methods. We evaluate our method on three major benchmarks of deep metric learning including CUB200-2011, Cars196, and Stanford Online Products, and achieve substantial improvements over popular metric learning methods with better interpretability. Code is available at https://github.com/wl-zhao/DIML

READ FULL TEXT

page 1

page 3

page 8

research
05/09/2022

Introspective Deep Metric Learning

This paper proposes an introspective deep metric learning (IDML) framewo...
research
09/20/2019

Deep Metric Learning using Similarities from Nonlinear Rank Approximations

In recent years, deep metric learning has achieved promising results in ...
research
11/17/2020

Towards Improved and Interpretable Deep Metric Learning via Attentive Grouping

Grouping has been commonly used in deep metric learning for computing di...
research
03/21/2023

Equiangular Basis Vectors

We propose Equiangular Basis Vectors (EBVs) for classification tasks. In...
research
03/28/2022

Attributable Visual Similarity Learning

This paper proposes an attributable visual similarity learning (AVSL) fr...
research
03/22/2021

Dynamic Metric Learning: Towards a Scalable Metric Space to Accommodate Multiple Semantic Scales

This paper introduces a new fundamental characteristic, , the dynamic ra...
research
06/01/2020

Multi-view Deep Features for Robust Facial Kinship Verification

Automatic kinship verification from facial images is an emerging researc...

Please sign up or login with your details

Forgot password? Click here to reset