VirtualPose: Learning Generalizable 3D Human Pose Models from Virtual Data

07/20/2022
by   Jiajun Su, et al.
0

While monocular 3D pose estimation seems to have achieved very accurate results on the public datasets, their generalization ability is largely overlooked. In this work, we perform a systematic evaluation of the existing methods and find that they get notably larger errors when tested on different cameras, human poses and appearance. To address the problem, we introduce VirtualPose, a two-stage learning framework to exploit the hidden "free lunch" specific to this task, i.e. generating infinite number of poses and cameras for training models at no cost. To that end, the first stage transforms images to abstract geometry representations (AGR), and then the second maps them to 3D poses. It addresses the generalization issue from two aspects: (1) the first stage can be trained on diverse 2D datasets to reduce the risk of over-fitting to limited appearance; (2) the second stage can be trained on diverse AGR synthesized from a large number of virtual cameras and poses. It outperforms the SOTA methods without using any paired images and 3D poses from the benchmarks, which paves the way for practical applications. Code is available at https://github.com/wkom/VirtualPose.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/07/2022

PoseGU: 3D Human Pose Estimation with Novel Human Pose Generator and Unbiased Learning

3D pose estimation has recently gained substantial interests in computer...
research
05/06/2021

PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation

Existing 3D human pose estimators suffer poor generalization performance...
research
07/29/2021

Improving Robustness and Accuracy via Relative Information Encoding in 3D Human Pose Estimation

Most of the existing 3D human pose estimation approaches mainly focus on...
research
03/27/2019

Dense Intrinsic Appearance Flow for Human Pose Transfer

We present a novel approach for the task of human pose transfer, which a...
research
03/30/2020

MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation

Cross view feature fusion is the key to address the occlusion problem in...
research
07/12/2022

Improving Domain Generalization by Learning without Forgetting: Application in Retail Checkout

Designing an automatic checkout system for retail stores at the human le...
research
11/03/2020

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild

Human head pose estimation in images has applications in many fields suc...

Please sign up or login with your details

Forgot password? Click here to reset