Where to Focus: Query Adaptive Matching for Instance Retrieval Using Convolutional Feature Maps

06/22/2016
by   Jiewei Cao, et al.
0

Instance retrieval requires one to search for images that contain a particular object within a large corpus. Recent studies show that using image features generated by pooling convolutional layer feature maps (CFMs) of a pretrained convolutional neural network (CNN) leads to promising performance for this task. However, due to the global pooling strategy adopted in those works, the generated image feature is less robust to image clutter and tends to be contaminated by the irrelevant image patterns. In this article, we alleviate this drawback by proposing a novel reranking algorithm using CFMs to refine the retrieval result obtained by existing methods. Our key idea, called query adaptive matching (QAM), is to first represent the CFMs of each image by a set of base regions which can be freely combined into larger regions-of-interest. Then the similarity between the query and a candidate image is measured by the best similarity score that can be attained by comparing the query feature and the feature pooled from a combined region. We show that the above procedure can be cast as an optimization problem and it can be solved efficiently with an off-the-shelf solver. Besides this general framework, we also propose two practical ways to create the base regions. One is based on the property of the CFM and the other one is based on a multi-scale spatial pyramid scheme. Through extensive experiments, we show that our reranking approaches bring substantial performance improvement and by applying them we can outperform the state of the art on several instance retrieval benchmarks.

READ FULL TEXT

page 3

page 4

page 7

page 9

research
04/15/2016

Bags of Local Convolutional Features for Scalable Instance Search

This work proposes a simple instance retrieval pipeline based on encodin...
research
11/27/2014

The Treasure beneath Convolutional Layers: Cross-convolutional-layer Pooling for Image Classification

A number of recent studies have shown that a Deep Convolutional Neural N...
research
11/05/2016

What Is the Best Practice for CNNs Applied to Visual Instance Retrieval?

Previous work has shown that feature maps of deep convolutional neural n...
research
10/04/2015

Cross-convolutional-layer Pooling for Image Recognition

Recent studies have shown that a Deep Convolutional Neural Network (DCNN...
research
08/30/2019

Fashion Retrieval via Graph Reasoning Networks on a Similarity Pyramid

Matching clothing images from customers and online shopping stores has r...
research
03/16/2022

Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting

Class-agnostic counting (CAC) aims to count all instances in a query ima...
research
06/19/2009

Finding Significant Subregions in Large Image Databases

Images have become an important data source in many scientific and comme...

Please sign up or login with your details

Forgot password? Click here to reset