Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid Structure from Motion with Detection of Temporally-Disjoint Rigidity

09/05/2019
by   Vladislav Golyanik, et al.
0

While dense non-rigid structure from motion (NRSfM) has been extensively studied from the perspective of the reconstructability problem over the recent years, almost no attempts have been undertaken to bring it into the practical realm. The reasons for the slow dissemination are the severe ill-posedness, high sensitivity to motion and deformation cues and the difficulty to obtain reliable point tracks in the vast majority of practical scenarios. To fill this gap, we propose a hybrid approach that extracts prior shape knowledge from an input sequence with NRSfM and uses it as a dynamic shape prior for sequential surface recovery in scenarios with recurrence. Our Dynamic Shape Prior Reconstruction (DSPR) method can be combined with existing dense NRSfM techniques while its energy functional is optimised with stochastic gradient descent at real-time rates for new incoming point tracks. The proposed versatile framework with a new core NRSfM approach outperforms several other methods in the ability to handle inaccurate and noisy point tracks, provided we have access to a representative (in terms of the deformation variety) image sequence. Comprehensive experiments highlight convergence properties and the accuracy of DSPR under different disturbing effects. We also perform a joint study of tracking and reconstruction and show applications to shape compression and heart reconstruction under occlusions. We achieve state-of-the-art metrics (accuracy and compression ratios) in different scenarios.

READ FULL TEXT

page 7

page 8

research
03/27/2018

HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model

Monocular dense 3D reconstruction of deformable objects is a hard ill-po...
research
12/20/2017

Accurate 3D Reconstruction of Dynamic Scenes from Monocular Image Sequences with Severe Occlusions

The paper introduces an accurate solution to dense orthographic Non-Rigi...
research
12/02/2020

Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction

We introduce Neural Deformation Graphs for globally-consistent deformati...
research
10/17/2017

Scalable Dense Monocular Surface Reconstruction

This paper reports on a novel template-free monocular non-rigid surface ...
research
04/27/2019

IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction

The majority of the existing methods for non-rigid 3D surface regression...
research
06/27/2017

Dense Non-rigid Structure-from-Motion Made Easy - A Spatial-Temporal Smoothness based Solution

This paper proposes a simple spatial-temporal smoothness based method fo...

Please sign up or login with your details

Forgot password? Click here to reset