On Background Bias in Deep Metric Learning

10/04/2022
by   Konstantin Kobs, et al.
2

Deep Metric Learning trains a neural network to map input images to a lower-dimensional embedding space such that similar images are closer together than dissimilar images. When used for item retrieval, a query image is embedded using the trained model and the closest items from a database storing their respective embeddings are returned as the most similar items for the query. Especially in product retrieval, where a user searches for a certain product by taking a photo of it, the image background is usually not important and thus should not influence the embedding process. Ideally, the retrieval process always returns fitting items for the photographed object, regardless of the environment the photo was taken in. In this paper, we analyze the influence of the image background on Deep Metric Learning models by utilizing five common loss functions and three common datasets. We find that Deep Metric Learning networks are prone to so-called background bias, which can lead to a severe decrease in retrieval performance when changing the image background during inference. We also show that replacing the background of images during training with random background images alleviates this issue. Since we use an automatic background removal method to do this background replacement, no additional manual labeling work and model changes are required while inference time stays the same. Qualitative and quantitative analyses, for which we introduce a new evaluation metric, confirm that models trained with replaced backgrounds attend more to the main object in the image, benefitting item retrieval systems.

READ FULL TEXT

page 2

page 6

research
05/05/2022

Do Different Deep Metric Learning Losses Lead to Similar Learned Features?

Recent studies have shown that many deep metric learning loss functions ...
research
11/30/2018

Making Classification Competitive for Deep Metric Learning

Deep metric learning aims to learn a function mapping image pixels to em...
research
03/05/2020

Embedding Expansion: Augmentation in Embedding Space for Deep Metric Learning

Learning the distance metric between pairs of samples has been studied f...
research
01/15/2018

Deep Metric Learning with BIER: Boosting Independent Embeddings Robustly

Learning similarity functions between image pairs with deep neural netwo...
research
04/08/2019

Improved Embeddings with Easy Positive Triplet Mining

Deep metric learning seeks to define an embedding where semantically sim...
research
10/11/2022

Large-to-small Image Resolution Asymmetry in Deep Metric Learning

Deep metric learning for vision is trained by optimizing a representatio...
research
05/24/2017

Generative Model with Coordinate Metric Learning for Object Recognition Based on 3D Models

Given large amount of real photos for training, Convolutional neural net...

Please sign up or login with your details

Forgot password? Click here to reset