On Symbiosis of Attribute Prediction and Semantic Segmentation

11/23/2019
by   Mahdi M. Kalayeh, et al.
13

In this paper, we propose to employ semantic segmentation to improve person-related attribute prediction. The core idea lies in the fact that the probability of an attribute to appear in an image is far from being uniform in the spatial domain. We build our attribute prediction model jointly with a deep semantic segmentation network. This harnesses the localization cues learned by the semantic segmentation to guide the attention of the attribute prediction to the regions where different attributes naturally show up. Therefore, in addition to prediction, we are able to localize the attributes despite merely having access to image-level labels (weak supervision) during training. We first propose semantic segmentation-based pooling and gating, respectively denoted as SSP and SSG. In the former, the estimated segmentation masks are used to pool the final activations of the attribute prediction network, from multiple semantically homogeneous regions. In SSG, the same idea is applied to the intermediate layers of the network. SSP and SSG, while effective, impose heavy memory utilization since each channel of the activations is pooled/gated with all the semantic segmentation masks. To circumvent this, we propose Symbiotic Augmentation (SA), where we learn only one mask per activation channel. SA allows the model to either pick one, or combine (weighted superposition) multiple semantic maps, in order to generate the proper mask for each channel. SA simultaneously applies the same mechanism to the reverse problem by leveraging output logits of attribute prediction to guide the semantic segmentation task. We evaluate our proposed methods for facial attributes on CelebA and LFWA datasets, while benchmarking WIDER Attribute and Berkeley Attributes of People for whole body attributes. Our proposed methods achieve superior results compared to the previous works.

READ FULL TEXT

page 2

page 3

page 6

page 7

page 8

page 12

page 13

page 15

research
03/20/2023

Generative Semantic Segmentation

We present Generative Semantic Segmentation (GSS), a generative learning...
research
03/04/2023

Exploit CAM by itself: Complementary Learning System for Weakly Supervised Semantic Segmentation

Weakly Supervised Semantic Segmentation (WSSS) with image-level labels h...
research
09/19/2022

A Causal Intervention Scheme for Semantic Segmentation of Quasi-periodic Cardiovascular Signals

Precise segmentation is a vital first step to analyze semantic informati...
research
12/19/2019

AANet: Attribute Attention Network for Person Re-Identifications

This paper proposes Attribute Attention Network (AANet), a new architect...
research
04/10/2021

Estimation of BMI from Facial Images using Semantic Segmentation based Region-Aware Pooling

Body-Mass-Index (BMI) conveys important information about one's life suc...
research
11/03/2021

HS3: Learning with Proper Task Complexity in Hierarchically Supervised Semantic Segmentation

While deeply supervised networks are common in recent literature, they t...
research
09/18/2019

Semantically Interpretable Activation Maps: what-where-how explanations within CNNs

A main issue preventing the use of Convolutional Neural Networks (CNN) i...

Please sign up or login with your details

Forgot password? Click here to reset