DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras

05/01/2021
by   Yang Zheng, et al.
22

We propose DeepMultiCap, a novel method for multi-person performance capture using sparse multi-view cameras. Our method can capture time varying surface details without the need of using pre-scanned template models. To tackle with the serious occlusion challenge for close interacting scenes, we combine a recently proposed pixel-aligned implicit function with parametric model for robust reconstruction of the invisible surface areas. An effective attention-aware module is designed to obtain the fine-grained geometry details from multi-view images, where high-fidelity results can be generated. In addition to the spatial attention method, for video inputs, we further propose a novel temporal fusion method to alleviate the noise and temporal inconsistencies for moving character reconstruction. For quantitative evaluation, we contribute a high quality multi-person dataset, MultiHuman, which consists of 150 static scenes with different levels of occlusions and ground truth 3D human models. Experimental results demonstrate the state-of-the-art performance of our method and the well generalization to real multiview video data, which outperforms the prior works by a large margin.

READ FULL TEXT

page 1

page 3

page 6

page 7

page 8

research
09/15/2021

Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering

In this paper, we aim at synthesizing a free-viewpoint video of an arbit...
research
02/28/2020

4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras

This paper contributes a novel realtime multi-person motion capture algo...
research
06/26/2023

TaiChi Action Capture and Performance Analysis with Multi-view RGB Cameras

Recent advances in computer vision and deep learning have influenced the...
research
03/19/2020

High Accuracy Face Geometry Capture using a Smartphone Video

What's the most accurate 3D model of your face you can obtain while sitt...
research
07/28/2020

Monocular Real-Time Volumetric Performance Capture

We present the first approach to volumetric performance capture and nove...
research
04/19/2021

Multi-person Implicit Reconstruction from a Single Image

We present a new end-to-end learning framework to obtain detailed and sp...
research
07/31/2023

Onboard View Planning of a Flying Camera for High Fidelity 3D Reconstruction of a Moving Actor

Capturing and reconstructing a human actor's motion is important for fil...

Please sign up or login with your details

Forgot password? Click here to reset