Represent, Compare, and Learn: A Similarity-Aware Framework for Class-Agnostic Counting

03/16/2022
by   Min Shi, et al.
0

Class-agnostic counting (CAC) aims to count all instances in a query image given few exemplars. A standard pipeline is to extract visual features from exemplars and match them with query images to infer object counts. Two essential components in this pipeline are feature representation and similarity metric. Existing methods either adopt a pretrained network to represent features or learn a new one, while applying a naive similarity metric with fixed inner product. We find this paradigm leads to noisy similarity matching and hence harms counting performance. In this work, we propose a similarity-aware CAC framework that jointly learns representation and similarity metric. We first instantiate our framework with a naive baseline called Bilinear Matching Network (BMNet), whose key component is a learnable bilinear similarity metric. To further embody the core of our framework, we extend BMNet to BMNet+ that models similarity from three aspects: 1) representing the instances via their self-similarity to enhance feature robustness against intra-class variations; 2) comparing the similarity dynamically to focus on the key patterns of each exemplar; 3) learning from a supervision signal to impose explicit constraints on matching results. Extensive experiments on a recent CAC dataset FSC147 show that our models significantly outperform state-of-the-art CAC approaches. In addition, we also validate the cross-dataset generality of BMNet and BMNet+ on a car counting dataset CARPK. Code is at tiny.one/BMNet

READ FULL TEXT

page 1

page 3

page 4

page 5

page 7

research
05/08/2023

Vision Transformer Off-the-Shelf: A Surprising Baseline for Few-Shot Class-Agnostic Counting

Class-agnostic counting (CAC) aims to count objects of interest from a q...
research
05/20/2022

Learning to Count Anything: Reference-less Class-agnostic Counting with Weak Supervision

Object counting is a seemingly simple task with diverse real-world appli...
research
11/01/2018

Class-Agnostic Counting

Nearly all existing counting methods are designed for a specific object ...
research
02/10/2023

GCNet: Probing Self-Similarity Learning for Generalized Counting Network

The class-agnostic counting (CAC) problem has caught increasing attentio...
research
06/22/2016

Where to Focus: Query Adaptive Matching for Instance Retrieval Using Convolutional Feature Maps

Instance retrieval requires one to search for images that contain a part...
research
10/28/2019

Neural Similarity Learning

Inner product-based convolution has been the founding stone of convoluti...
research
06/27/2020

Counting Out Time: Class Agnostic Video Repetition Counting in the Wild

We present an approach for estimating the period with which an action is...

Please sign up or login with your details

Forgot password? Click here to reset