Scene Graph Embeddings Using Relative Similarity Supervision

04/06/2021
by   Paridhi Maheshwari, et al.
0

Scene graphs are a powerful structured representation of the underlying content of images, and embeddings derived from them have been shown to be useful in multiple downstream tasks. In this work, we employ a graph convolutional network to exploit structure in scene graphs and produce image embeddings useful for semantic image retrieval. Different from classification-centric supervision traditionally available for learning image representations, we address the task of learning from relative similarity labels in a ranking context. Rooted within the contrastive learning paradigm, we propose a novel loss function that operates on pairs of similar and dissimilar images and imposes relative ordering between them in embedding space. We demonstrate that this Ranking loss, coupled with an intuitive triple sampling strategy, leads to robust representations that outperform well-known contrastive losses on the retrieval task. In addition, we provide qualitative evidence of how retrieved results that utilize structured scene information capture the global context of the scene, different from visual similarity search.

READ FULL TEXT

page 2

page 5

research
04/02/2023

Learning Similarity between Scene Graphs and Images with Transformers

Scene graph generation is conventionally evaluated by (mean) Recall@K, w...
research
01/27/2022

Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives

This paper introduces Ranking Info Noise Contrastive Estimation (RINCE),...
research
09/19/2019

Triplet-Aware Scene Graph Embeddings

Scene graphs have become an important form of structured knowledge for t...
research
04/28/2023

SGAligner : 3D Scene Alignment with Scene Graphs

Building 3D scene graphs has recently emerged as a topic in scene repres...
research
12/29/2020

Image-to-Image Retrieval by Learning Similarity between Scene Graphs

As a scene graph compactly summarizes the high-level content of an image...
research
09/18/2023

Traffic Scene Similarity: a Graph-based Contrastive Learning Approach

Ensuring validation for highly automated driving poses significant obsta...
research
06/20/2022

Self-Supervised Consistent Quantization for Fully Unsupervised Image Retrieval

Unsupervised image retrieval aims to learn an efficient retrieval system...

Please sign up or login with your details

Forgot password? Click here to reset