SiCloPe: Silhouette-Based Clothed People

by   Ryota Natsume, et al.

We introduce a new silhouette-based representation for modeling clothed human bodies using deep generative models. Our method can reconstruct a complete and textured 3D model of a person wearing clothes from a single input picture. Inspired by the visual hull algorithm, our implicit representation uses 2D silhouettes and 3D joints of a body pose to describe the immense shape complexity and variations of clothed people. Given a segmented 2D silhouette of a person and its inferred 3D joints from the input picture, we first synthesize consistent silhouettes from novel view points around the subject. The synthesized silhouettes, which are the most consistent with the input segmentation are fed into a deep visual hull algorithm for robust 3D shape prediction. We then infer the texture of the subject's back view using the frontal image and segmentation mask as input to a conditional generative adversarial network. Our experiments demonstrate that our silhouette-based model is an effective representation and the appearance of the back view can be predicted reliably using an image-to-image translation network. While classic methods based on parametric models often fail for single-view images of subjects with challenging clothing, our approach can still produce successful results, which are comparable to those obtained from multi-view input.


page 3

page 4

page 5

page 6

page 7

page 8


360-Degree Textures of People in Clothing from a Single Image

In this paper we predict a full 3D avatar of a person from a single imag...

RIN: Textured Human Model Recovery and Imitation with a Single Image

Human imitation has become topical recently, driven by GAN's ability to ...

Specular-to-Diffuse Translation for Multi-View Reconstruction

Most multi-view 3D reconstruction algorithms, especially when shape-from...

Sem2NeRF: Converting Single-View Semantic Masks to Neural Radiance Fields

Image translation and manipulation have gain increasing attention along ...

Vid2Actor: Free-viewpoint Animatable Person Synthesis from Video in the Wild

Given an "in-the-wild" video of a person, we reconstruct an animatable m...

Learning to Reconstruct People in Clothing from a Single RGB Camera

We present a learning-based model to infer the personalized 3D shape of ...

ZIGNeRF: Zero-shot 3D Scene Representation with Invertible Generative Neural Radiance Fields

Generative Neural Radiance Fields (NeRFs) have demonstrated remarkable p...

Code Repositories


High-Resolution 3D Human Digitization from A Single Image.

view repo


This repository contains the code for the paper "PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization"

view repo

Please sign up or login with your details

Forgot password? Click here to reset