PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization

05/13/2019
by   Shunsuke Saito, et al.
23

We introduce Pixel-aligned Implicit Function (PIFu), a highly effective implicit representation that locally aligns pixels of 2D images with the global context of their corresponding 3D object. Using PIFu, we propose an end-to-end deep learning method for digitizing highly detailed clothed humans that can infer both 3D surface and texture from a single image, and optionally, multiple input images. Highly intricate shapes, such as hairstyles, clothing, as well as their variations and deformations can be digitized in a unified way. Compared to existing representations used for 3D deep learning, PIFu can produce high-resolution surfaces including largely unseen regions such as the back of a person. In particular, it is memory efficient unlike the voxel representation, can handle arbitrary topology, and the resulting surface is spatially aligned with the input image. Furthermore, while previous techniques are designed to process either a single image or multiple views, PIFu extends naturally to arbitrary number of views. We demonstrate high-resolution and robust reconstructions on real world images from the DeepFashion dataset, which contains a variety of challenging clothing types. Our method achieves state-of-the-art performance on a public benchmark and outperforms the prior work for clothed human digitization from a single image.

READ FULL TEXT

page 1

page 4

page 6

page 7

page 8

page 12

page 13

page 14

research
04/01/2020

PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization

Recent advances in image-based 3D human shape estimation have been drive...
research
04/19/2021

Multi-person Implicit Reconstruction from a Single Image

We present a new end-to-end learning framework to obtain detailed and sp...
research
06/14/2022

Reconstructing vehicles from orthographic drawings using deep neural networks

This paper explores the current state-of-the-art of object reconstructio...
research
12/13/2022

Structured 3D Features for Reconstructing Relightable and Animatable Avatars

We introduce Structured 3D Features, a model based on a novel implicit 3...
research
06/29/2021

TUCaN: Progressively Teaching Colourisation to Capsules

Automatic image colourisation is the computer vision research path that ...
research
08/25/2022

Learning Continuous Implicit Representation for Near-Periodic Patterns

Near-Periodic Patterns (NPP) are ubiquitous in man-made scenes and are c...
research
11/30/2020

Vehicle Reconstruction and Texture Estimation Using Deep Implicit Semantic Template Mapping

We introduce VERTEX, an effective solution to recover 3D shape and intri...

Please sign up or login with your details

Forgot password? Click here to reset