DeepAI AI Chat
Log In Sign Up

Multi-task human analysis in still images: 2D/3D pose, depth map, and multi-part segmentation

by   Daniel Sanchez, et al.

While many individual tasks in the domain of human analysis have recently received an accuracy boost from deep learning approaches, multi-task learning has mostly been ignored due to a lack of data. New synthetic datasets are being released, filling this gap with synthetic generated data. In this work, we analyze four related human analysis tasks in still images in a multi-task scenario by leveraging such datasets. Specifically, we study the correlation of 2D/3D pose estimation, body part segmentation and full-body depth estimation. These tasks are learned via the well-known Stacked Hourglass module such that each of the task-specific streams shares information with the others. The main goal is to analyze how training together these four related tasks can benefit each individual task for a better generalization. Results on the newly released SURREAL dataset show that all four tasks benefit from the multi-task approach, but with different combinations of tasks: while combining all four tasks improves 2D pose estimation the most, 2D pose improves neither 3D pose nor full-body depth estimation. On the other hand 2D parts segmentation can benefit from 2D pose but not from 3D pose. In all cases, as expected, the maximum improvement is achieved on those human body parts that show more variability in terms of spatial distribution, appearance and shape, e.g. wrists and ankles.


page 1

page 2

page 3

page 7


Human Pose Estimation from Depth Images via Inference Embedded Multi-task Learning

Human pose estimation (i.e., locating the body parts / joints of a perso...

Object-Centric Multi-Task Learning for Human Instances

Human is one of the most essential classes in visual recognition tasks s...

Instance-Level Task Parameters: A Robust Multi-task Weighting Framework

Recent works have shown that deep neural networks benefit from multi-tas...

Visual Person Understanding through Multi-Task and Multi-Dataset Learning

We address the problem of learning a single model for person re-identifi...

PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data

In comparison with person re-identification (ReID), which has been widel...

Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap

This work presents Spacecraft Pose Network v2 (SPNv2), a Convolutional N...

Effect of Gender, Pose and Camera Distance on Human Body Dimensions Estimation

Human Body Dimensions Estimation (HBDE) is a task that an intelligent ag...