Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction

03/30/2021
by   Shanyan Guan, et al.
0

This paper considers a new problem of adapting a pre-trained model of human mesh reconstruction to out-of-domain streaming videos. However, most previous methods based on the parametric SMPL model <cit.> underperform in new domains with unexpected, domain-specific attributes, such as camera parameters, lengths of bones, backgrounds, and occlusions. Our general idea is to dynamically fine-tune the source model on test video streams with additional temporal constraints, such that it can mitigate the domain gaps without over-fitting the 2D information of individual test frames. A subsequent challenge is how to avoid conflicts between the 2D and temporal constraints. We propose to tackle this problem using a new training algorithm named Bilevel Online Adaptation (BOA), which divides the optimization process of overall multi-objective into two steps of weight probe and weight update in a training iteration. We demonstrate that BOA leads to state-of-the-art results on two human mesh reconstruction benchmarks.

READ FULL TEXT

page 1

page 6

page 7

research
11/07/2021

Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation

We consider a new problem of adapting a human mesh reconstruction model ...
research
08/12/2023

Cyclic Test-Time Adaptation on Monocular Video for 3D Human Mesh Reconstruction

Despite recent advances in 3D human mesh reconstruction, domain gap betw...
research
03/20/2019

Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction

In this paper, we address the problem of 3D object mesh reconstruction f...
research
02/09/2023

Accurate and Interpretable Solution of the Inverse Rig for Realistic Blendshape Models with Quadratic Corrective Terms

We propose a new model-based algorithm solving the inverse rig problem i...
research
12/21/2020

Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction

Deep learning-solutions for hand-object 3D pose and shape estimation are...
research
12/17/2020

Human Mesh Recovery from Multiple Shots

Videos from edited media like movies are a useful, yet under-explored so...

Please sign up or login with your details

Forgot password? Click here to reset