PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo

03/22/2022
by   Jiachen Liu, et al.
0

We present a novel framework named PlaneMVS for 3D plane reconstruction from multiple input views with known camera poses. Most previous learning-based plane reconstruction methods reconstruct 3D planes from single images, which highly rely on single-view regression and suffer from depth scale ambiguity. In contrast, we reconstruct 3D planes with a multi-view-stereo (MVS) pipeline that takes advantage of multi-view geometry. We decouple plane reconstruction into a semantic plane detection branch and a plane MVS branch. The semantic plane detection branch is based on a single-view plane detection framework but with differences. The plane MVS branch adopts a set of slanted plane hypotheses to replace conventional depth hypotheses to perform plane sweeping strategy and finally learns pixel-level plane parameters and its planar depth map. We present how the two branches are learned in a balanced way, and propose a soft-pooling loss to associate the outputs of the two branches and make them benefit from each other. Extensive experiments on various indoor datasets show that PlaneMVS significantly outperforms state-of-the-art (SOTA) single-view plane reconstruction methods on both plane detection and 3D geometry metrics. Our method even outperforms a set of SOTA learning-based MVS methods thanks to the learned plane priors. To the best of our knowledge, this is the first work on 3D plane reconstruction within an end-to-end MVS framework.

READ FULL TEXT

page 7

page 8

page 10

page 11

page 12

page 13

page 14

research
04/13/2021

PHI-MVS: Plane Hypothesis Inference Multi-view Stereo for Large-Scale Scene Reconstruction

PatchMatch based Multi-view Stereo (MVS) algorithms have achieved great ...
research
06/17/2020

Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction

3D reconstruction from a single RGB image is a challenging problem in co...
research
03/25/2022

Microstructure Surface Reconstruction from SEM Images: An Alternative to Digital Image Correlation (DIC)

We reconstruct a 3D model of the surface of a material undergoing fatigu...
research
12/12/2021

MVLayoutNet:3D layout reconstruction with multi-view panoramas

We present MVLayoutNet, an end-to-end network for holistic 3D reconstruc...
research
05/02/2017

Active Image-based Modeling

We seek to automate the data capturing process in image-based modeling, ...
research
07/17/2020

A direct reconstruction algorithm for the anisotropic inverse conductivity problem based on Calderón's method in the plane

A direct reconstruction algorithm based on Calderón's linearization meth...
research
03/10/2023

Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstruction

The Multiplane Image (MPI), containing a set of fronto-parallel RGBA lay...

Please sign up or login with your details

Forgot password? Click here to reset