Fine-grained Recognition with Learnable Semantic Data Augmentation

by   Yifan Pu, et al.

Fine-grained image recognition is a longstanding computer vision challenge that focuses on differentiating objects belonging to multiple subordinate categories within the same meta-category. Since images belonging to the same meta-category usually share similar visual appearances, mining discriminative visual cues is the key to distinguishing fine-grained categories. Although commonly used image-level data augmentation techniques have achieved great success in generic image classification problems, they are rarely applied in fine-grained scenarios, because their random editing-region behavior is prone to destroy the discriminative visual cues residing in the subtle regions. In this paper, we propose diversifying the training data at the feature-level to alleviate the discriminative region loss problem. Specifically, we produce diversified augmented samples by translating image features along semantically meaningful directions. The semantic directions are estimated with a covariance prediction network, which predicts a sample-wise covariance matrix to adapt to the large intra-class variation inherent in fine-grained images. Furthermore, the covariance prediction network is jointly optimized with the classification network in a meta-learning manner to alleviate the degenerate solution problem. Experiments on four competitive fine-grained recognition benchmarks (CUB-200-2011, Stanford Cars, FGVC Aircrafts, NABirds) demonstrate that our method significantly improves the generalization performance on several popular classification networks (e.g., ResNets, DenseNets, EfficientNets, RegNets and ViT). Combined with a recently proposed method, our semantic data augmentation approach achieves state-of-the-art performance on the CUB-200-2011 dataset. The source code will be released.


page 1

page 2

page 3

page 8

page 10

page 14


Attribute Mix: Semantic Data Augmentation for Fine Grained Recognition

Collecting fine-grained labels usually requires expert-level domain know...

Re-rank Coarse Classification with Local Region Enhanced Features for Fine-Grained Image Recognition

Fine-grained image recognition is very challenging due to the difficulty...

ELFIS: Expert Learning for Fine-grained Image Recognition Using Subsets

Fine-Grained Visual Recognition (FGVR) tackles the problem of distinguis...

Hierarchical CVAE for Fine-Grained Hate Speech Classification

Existing work on automated hate speech detection typically focuses on bi...

ScoreNet: Learning Non-Uniform Attention and Augmentation for Transformer-Based Histopathological Image Classification

Progress in digital pathology is hindered by high-resolution images and ...

MetaFormer: A Unified Meta Framework for Fine-Grained Recognition

Fine-Grained Visual Classification(FGVC) is the task that requires recog...

Knowledge Transfer Based Fine-grained Visual Classification

Fine-grained visual classification (FGVC) aims to distinguish the sub-cl...

Please sign up or login with your details

Forgot password? Click here to reset