Learning Compositional Neural Information Fusion for Human Parsing

by   Wenguan Wang, et al.

This work proposes to combine neural networks with the compositional hierarchy of human bodies for efficient and complete human parsing. We formulate the approach as a neural information fusion framework. Our model assembles the information from three inference processes over the hierarchy: direct inference (directly predicting each part of a human body using image information), bottom-up inference (assembling knowledge from constituent parts), and top-down inference (leveraging context from parent nodes). The bottom-up and top-down inferences explicitly model the compositional and decompositional relations in human bodies, respectively. In addition, the fusion of multi-source information is conditioned on the inputs, i.e., by estimating and considering the confidence of the sources. The whole model is end-to-end differentiable, explicitly modeling information flows and structures. Our approach is extensively evaluated on four popular datasets, outperforming the state-of-the-arts in all cases, with a fast processing speed of 23fps. Our code and results have been released to help ease future research in this direction.


page 1

page 3

page 5

page 8


Devil in the Details: Towards Accurate Single and Multiple Human Parsing

Human parsing has received considerable interest due to its wide applica...

Learning compositional structures for semantic graph parsing

AM dependency parsing is a method for neural semantic graph parsing that...

Compositional Generalization for Primitive Substitutions

Compositional generalization is a basic mechanism in human language lear...

Understanding Robust Generalization in Learning Regular Languages

A key feature of human intelligence is the ability to generalize beyond ...

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

To address the challenging task of instance-aware human part parsing, a ...

Learning Visual Symbols for Parsing Human Poses in Images

Parsing human poses in images is fundamental in extracting critical visu...

Learning Associative Inference Using Fast Weight Memory

Humans can quickly associate stimuli to solve problems in novel contexts...