Instance-level Image Retrieval using Reranking Transformers

03/22/2021
by   Fuwen Tan, et al.
0

Instance-level image retrieval is the task of searching in a large database for images that match an object in a query image. To address this task, systems usually rely on a retrieval step that uses global image descriptors, and a subsequent step that performs domain-specific refinements or reranking by leveraging operations such as geometric verification based on local features. In this work, we propose Reranking Transformers (RRTs) as a general model to incorporate both local and global features to rerank the matching images in a supervised fashion and thus replace the relatively expensive process of geometric verification. RRTs are lightweight and can be easily parallelized so that reranking a set of top matching results can be performed in a single forward-pass. We perform extensive experiments on the Revisited Oxford and Paris datasets, and the Google Landmark v2 dataset, showing that RRTs outperform previous reranking approaches while using much fewer local descriptors. Moreover, we demonstrate that, unlike existing approaches, RRTs can be optimized jointly with the feature extractor, which can lead to feature representations tailored to downstream tasks and further accuracy improvements. Training code and pretrained models will be made public.

READ FULL TEXT

page 3

page 8

page 11

page 12

research
12/19/2016

Large-Scale Image Retrieval with Attentive Deep Local Features

We propose an attentive local feature descriptor suitable for large-scal...
research
04/05/2016

Deep Image Retrieval: Learning global representations for image search

We propose a novel approach for instance-level image retrieval. It produ...
research
02/10/2021

Training Vision Transformers for Image Retrieval

Transformers have shown outstanding results for natural language underst...
research
10/07/2021

Efficient large-scale image retrieval with deep feature orthogonality and Hybrid-Swin-Transformers

We present an efficient end-to-end pipeline for largescale landmark reco...
research
01/14/2020

Unifying Deep Local and Global Features for Efficient Image Search

A key challenge in large-scale image retrieval problems is the trade-off...
research
08/13/2020

Predicting Visual Overlap of Images Through Interpretable Non-Metric Box Embeddings

To what extent are two images picturing the same 3D surfaces? Even when ...
research
03/13/2019

Towards Accurate Camera Geopositioning by Image Matching

In this work, we present a camera geopositioning system based on matchin...

Please sign up or login with your details

Forgot password? Click here to reset