Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN

11/20/2021
by   Zhenyu Xie, et al.
1

Image-based virtual try-on is one of the most promising applications of human-centric image generation due to its tremendous real-world potential. Yet, as most try-on approaches fit in-shop garments onto a target person, they require the laborious and restrictive construction of a paired training dataset, severely limiting their scalability. While a few recent works attempt to transfer garments directly from one person to another, alleviating the need to collect paired datasets, their performance is impacted by the lack of paired (supervised) information. In particular, disentangling style and spatial information of the garment becomes a challenge, which existing methods either address by requiring auxiliary data or extensive online optimization procedures, thereby still inhibiting their scalability. To achieve a scalable virtual try-on system that can transfer arbitrary garments between a source and a target person in an unsupervised manner, we thus propose a texture-preserving end-to-end network, the PAtch-routed SpaTially-Adaptive GAN (PASTA-GAN), that facilitates real-world unpaired virtual try-on. Specifically, to disentangle the style and spatial information of each garment, PASTA-GAN consists of an innovative patch-routed disentanglement module for successfully retaining garment texture and shape characteristics. Guided by the source person keypoints, the patch-routed disentanglement module first decouples garments into normalized patches, thus eliminating the inherent spatial information of the garment, and then reconstructs the normalized patches to the warped garment complying with the target person pose. Given the warped garment, PASTA-GAN further introduces novel spatially-adaptive residual blocks that guide the generator to synthesize more realistic garment details.

READ FULL TEXT

page 2

page 8

page 9

research
07/27/2022

PASTA-GAN++: A Versatile Framework for High-Resolution Unpaired Virtual Try-on

Image-based virtual try-on is one of the most promising applications of ...
research
03/29/2022

Dressing in the Wild by Watching Dance Videos

While significant progress has been made in garment transfer, one of the...
research
07/20/2018

Toward Characteristic-Preserving Image-based Virtual Try-On Network

Image-based virtual try-on systems for fitting a new in-shop clothes int...
research
11/19/2022

Single Stage Multi-Pose Virtual Try-On

Multi-pose virtual try-on (MPVTON) aims to fit a target garment onto a p...
research
04/01/2020

LGVTON: A Landmark Guided Approach to Virtual Try-On

We address the problem of image based virtual try-on (VTON), where the g...
research
01/06/2021

TryOnGAN: Body-Aware Try-On via Layered Interpolation

Given a pair of images-target person and garment on another person-we au...
research
09/10/2021

Per Garment Capture and Synthesis for Real-time Virtual Try-on

Virtual try-on is a promising application of computer graphics and human...

Please sign up or login with your details

Forgot password? Click here to reset