KE-RCNN: Unifying Knowledge based Reasoning into Part-level Attribute Parsing

06/21/2022
by   Xuanhan Wang, et al.
0

Part-level attribute parsing is a fundamental but challenging task, which requires the region-level visual understanding to provide explainable details of body parts. Most existing approaches address this problem by adding a regional convolutional neural network (RCNN) with an attribute prediction head to a two-stage detector, in which attributes of body parts are identified from local-wise part boxes. However, local-wise part boxes with limit visual clues (i.e., part appearance only) lead to unsatisfying parsing results, since attributes of body parts are highly dependent on comprehensive relations among them. In this article, we propose a Knowledge Embedded RCNN (KE-RCNN) to identify attributes by leveraging rich knowledges, including implicit knowledge (e.g., the attribute “above-the-hip” for a shirt requires visual/geometry relations of shirt-hip) and explicit knowledge (e.g., the part of “shorts” cannot have the attribute of “hoodie” or “lining”). Specifically, the KE-RCNN consists of two novel components, i.e., Implicit Knowledge based Encoder (IK-En) and Explicit Knowledge based Decoder (EK-De). The former is designed to enhance part-level representation by encoding part-part relational contexts into part boxes, and the latter one is proposed to decode attributes with a guidance of prior knowledge about part-attribute relations. In this way, the KE-RCNN is plug-and-play, which can be integrated into any two-stage detectors, e.g., Attribute-RCNN, Cascade-RCNN, HRNet based RCNN and SwinTransformer based RCNN. Extensive experiments conducted on two challenging benchmarks, e.g., Fashionpedia and Kinetics-TPS, demonstrate the effectiveness and generalizability of the KE-RCNN. In particular, it achieves higher improvements over all existing methods, reaching around 3 Fashionpedia and around 4

READ FULL TEXT

page 1

page 3

page 8

page 9

page 11

page 15

research
11/19/2018

CA3Net: Contextual-Attentional Attribute-Appearance Network for Person Re-Identification

Person re-identification aims to identify the same pedestrian across non...
research
10/10/2019

Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization

Pedestrian attribute recognition has been an emerging research topic in ...
research
12/08/2014

Actions and Attributes from Wholes and Parts

We investigate the importance of parts for the tasks of action and attri...
research
02/27/2019

Attributes-aided Part Detection and Refinement for Person Re-identification

Person attributes are often exploited as mid-level human semantic inform...
research
03/29/2022

3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow

Reconstructing 3D shape from a single 2D image is a challenging task, wh...
research
12/15/2022

Body-Part Joint Detection and Association via Extended Object Representation

The detection of human body and its related parts (e.g., face, head or h...
research
08/23/2023

CIParsing: Unifying Causality Properties into Multiple Human Parsing

Existing methods of multiple human parsing (MHP) apply statistical model...

Please sign up or login with your details

Forgot password? Click here to reset