Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning

11/25/2022
by   Zaiyu Huang, et al.
1

In this paper, we target image-based person-to-person virtual try-on in the presence of diverse poses and large viewpoint variations. Existing methods are restricted in this setting as they estimate garment warping flows mainly based on 2D poses and appearance, which omits the geometric prior of the 3D human body shape. Moreover, current garment warping methods are confined to localized regions, which makes them ineffective in capturing long-range dependencies and results in inferior flows with artifacts. To tackle these issues, we present 3D-aware global correspondences, which are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies. Particularly, given an image pair depicting the source and target person, (a) we first obtain their pose-aware and high-level representations via two encoders, and introduce a coarse-to-fine decoder with multiple refinement modules to predict the pixel-wise global correspondence. (b) 3D parametric human models inferred from images are incorporated as priors to regularize the correspondence refinement process so that our flows can be 3D-aware and better handle variations of pose and viewpoint. (c) Finally, an adversarial generator takes the garment warped by the 3D-aware flow, and the image of the target person as inputs, to synthesize the photo-realistic try-on result. Extensive experiments on public benchmarks and our HardPose test set demonstrate the superiority of our method against the SOTA try-on approaches.

READ FULL TEXT

page 2

page 7

page 9

research
03/30/2023

Learning Garment DensePose for Robust Warping in Virtual Try-On

Virtual try-on, i.e making people virtually try new garments, is an acti...
research
10/03/2022

Fill in Fabrics: Body-Aware Self-Supervised Inpainting for Image-Based Virtual Try-On

Previous virtual try-on methods usually focus on aligning a clothing ite...
research
03/24/2023

GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning

Image-based Virtual Try-ON aims to transfer an in-shop garment onto a sp...
research
05/29/2017

Pose-Aware Person Recognition

Person recognition methods that use multiple body regions have shown sig...
research
08/17/2022

Significance of Skeleton-based Features in Virtual Try-On

The idea of Virtual Try-ON (VTON) benefits e-retailing by giving an user...
research
09/14/2021

ZFlow: Gated Appearance Flow-based Virtual Try-on with 3D Priors

Image-based virtual try-on involves synthesizing perceptually convincing...
research
09/13/2021

Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN

We present an algorithm for re-rendering a person from a single image un...

Please sign up or login with your details

Forgot password? Click here to reset