Low-rank Bilinear Pooling for Fine-Grained Classification

11/16/2016
by   Shu Kong, et al.
0

Pooling second-order local feature statistics to form a high-dimensional bilinear feature has been shown to achieve state-of-the-art performance on a variety of fine-grained classification tasks. To address the computational demands of high feature dimensionality, we propose to represent the covariance features as a matrix and apply a low-rank bilinear classifier. The resulting classifier can be evaluated without explicitly computing the bilinear feature map which allows for a large reduction in the compute time as well as decreasing the effective number of parameters to be learned. To further compress the model, we propose classifier co-decomposition that factorizes the collection of bilinear classifiers into a common factor and compact per-class terms. The co-decomposition idea can be deployed through two convolutional layers and trained in an end-to-end architecture. We suggest a simple yet effective initialization that avoids explicitly first training and factorizing the larger bilinear classifiers. Through extensive experiments, we show that our model achieves state-of-the-art performance on several public datasets for fine-grained classification trained with only category labels. Importantly, our final model is an order of magnitude smaller than the recently proposed compact bilinear model, and three orders smaller than the standard bilinear CNN model.

READ FULL TEXT

page 7

page 8

research
06/03/2019

Low-rank Random Tensor for Bilinear Pooling

Bilinear pooling is capable of extracting high-order information from da...
research
08/04/2019

Low-Rank Pairwise Alignment Bilinear Network For Few-Shot Fine-Grained Image Classification

Deep neural networks have demonstrated advanced abilities on various vis...
research
02/20/2018

MoNet: Moments Embedding Network

Bilinear pooling has been recently proposed as a feature encoding layer,...
research
03/30/2020

Multi-Objective Matrix Normalization for Fine-grained Visual Recognition

Bilinear pooling achieves great success in fine-grained visual recogniti...
research
06/05/2019

Compact Approximation for Polynomial of Covariance Feature

Covariance pooling is a feature pooling method with good classification ...
research
12/18/2020

Temporal Bilinear Encoding Network of Audio-Visual Features at Low Sampling Rates

Current deep learning based video classification architectures are typic...
research
04/29/2015

Bilinear CNNs for Fine-grained Visual Recognition

We present a simple and effective architecture for fine-grained visual r...

Please sign up or login with your details

Forgot password? Click here to reset