DeepAI AI Chat
Log In Sign Up

Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction

by   Shanyan Guan, et al.
Shanghai Jiao Tong University

This paper considers a new problem of adapting a pre-trained model of human mesh reconstruction to out-of-domain streaming videos. However, most previous methods based on the parametric SMPL model <cit.> underperform in new domains with unexpected, domain-specific attributes, such as camera parameters, lengths of bones, backgrounds, and occlusions. Our general idea is to dynamically fine-tune the source model on test video streams with additional temporal constraints, such that it can mitigate the domain gaps without over-fitting the 2D information of individual test frames. A subsequent challenge is how to avoid conflicts between the 2D and temporal constraints. We propose to tackle this problem using a new training algorithm named Bilevel Online Adaptation (BOA), which divides the optimization process of overall multi-objective into two steps of weight probe and weight update in a training iteration. We demonstrate that BOA leads to state-of-the-art results on two human mesh reconstruction benchmarks.


page 1

page 6

page 7


Out-of-Domain Human Mesh Reconstruction via Dynamic Bilevel Online Adaptation

We consider a new problem of adapting a human mesh reconstruction model ...

Cyclic Test-Time Adaptation on Monocular Video for 3D Human Mesh Reconstruction

Despite recent advances in 3D human mesh reconstruction, domain gap betw...

Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction

In this paper, we address the problem of 3D object mesh reconstruction f...

Unsupervised Domain Adaptation with Temporal-Consistent Self-Training for 3D Hand-Object Joint Reconstruction

Deep learning-solutions for hand-object 3D pose and shape estimation are...

Human Mesh Recovery from Multiple Shots

Videos from edited media like movies are a useful, yet under-explored so...

Code Repositories


Bilevel Online Adaptation for Human Mesh Reconstruction

view repo