Deep Imbalanced Attribute Classification using Visual Attention Aggregation

07/10/2018
by   Nikolaos Sarafianos, et al.
2

For many computer vision applications such as image description and human identification, recognizing the visual attributes of humans is an essential yet challenging problem. Its challenges originate from its multi-label nature, the large underlying class imbalance and the lack of spatial annotations. Existing methods follow either a computer vision approach while failing to account for class imbalance, or explore machine learning solutions, which disregard the spatial and semantic relations that exist in the images. With that in mind, we propose an effective method that extracts and aggregates visual attention masks at different scales. We introduce a loss function to handle class imbalance both at class and at an instance level and further demonstrate that penalizing attention masks with high prediction variance accounts for the weak supervision of the attention mechanism. By identifying and addressing these challenges, we achieve state-of-the-art results with a simple attention mechanism in both PETA and WIDER-Attribute datasets without additional context or side information.

READ FULL TEXT

page 2

page 6

page 11

page 14

research
11/27/2022

ReGrAt: Regularization in Graphs using Attention to handle class imbalance

Node classification is an important task to solve in graph-based learnin...
research
07/20/2020

Relatable Clothing: Detecting Visual Relationships between People and Clothing

Detecting visual relationships between people and clothing in an image h...
research
09/05/2017

Multi-label Class-imbalanced Action Recognition in Hockey Videos via 3D Convolutional Neural Networks

Automatic analysis of the video is one of most complex problems in the f...
research
06/17/2021

Learning to Predict Visual Attributes in the Wild

Visual attributes constitute a large portion of information contained in...
research
08/05/2021

Residual Attention: A Simple but Effective Method for Multi-Label Recognition

Multi-label image recognition is a challenging computer vision task of p...
research
11/26/2019

Distraction-Aware Feature Learning for Human Attribute Recognition via Coarse-to-Fine Attention Mechanism

Recently, Human Attribute Recognition (HAR) has become a hot topic due t...
research
07/26/2020

U2-ONet: A Two-level Nested Octave U-structure with Multiscale Attention Mechanism for Moving Instances Segmentation

Most scenes in practical applications are dynamic scenes containing movi...

Please sign up or login with your details

Forgot password? Click here to reset