DeepAI AI Chat
Log In Sign Up

PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision

by   Kehong Gong, et al.
National University of Singapore

Existing self-supervised 3D human pose estimation schemes have largely relied on weak supervisions like consistency loss to guide the learning, which, inevitably, leads to inferior results in real-world scenarios with unseen poses. In this paper, we propose a novel self-supervised approach that allows us to explicitly generate 2D-3D pose pairs for augmenting supervision, through a self-enhancing dual-loop learning framework. This is made possible via introducing a reinforcement-learning-based imitator, which is learned jointly with a pose estimator alongside a pose hallucinator; the three components form two loops during the training process, complementing and strengthening one another. Specifically, the pose estimator transforms an input 2D pose sequence to a low-fidelity 3D output, which is then enhanced by the imitator that enforces physical constraints. The refined 3D poses are subsequently fed to the hallucinator for producing even more diverse data, which are, in turn, strengthened by the imitator and further utilized to train the pose estimator. Such a co-evolution scheme, in practice, enables training a pose estimator on self-generated motion data without relying on any given 3D data. Extensive experiments across various benchmarks demonstrate that our approach yields encouraging results significantly outperforming the state of the art and, in some cases, even on par with results of fully-supervised methods. Notably, it achieves 89.1 evaluation setup, improving upon the previous best self-supervised methods by 8.6


page 4

page 8


Unsupervised 3D Pose Estimation with Geometric Self-Supervision

We present an unsupervised learning approach to recover 3D human pose fr...

3D Human Pose Machines with Self-supervised Learning

Driven by recent computer vision and robotic applications, recovering 3D...

Self-supervised 3D Human Pose Estimation from a Single Image

We propose a new self-supervised method for predicting 3D human body pos...

Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation

Existing 3D human pose estimation models suffer performance drop when ap...

FetusMap: Fetal Pose Estimation in 3D Ultrasound

The 3D ultrasound (US) entrance inspires a multitude of automated prenat...

Self-Supervised Learning of Image Scale and Orientation

We study the problem of learning to assign a characteristic pose, i.e., ...

A Correct-and-Certify Approach to Self-Supervise Object Pose Estimators via Ensemble Self-Training

Real-world robotics applications demand object pose estimation methods t...

Code Repositories


[CVPR 2022] PoseTriplet: Co-evolving 3D Human Pose Estimation, Imitation, and Hallucination under Self-supervision (Oral)

view repo