GARNet: Global-Aware Multi-View 3D Reconstruction Network and the Cost-Performance Tradeoff

11/04/2022
by   Zhenwei Zhu, et al.
4

Deep learning technology has made great progress in multi-view 3D reconstruction tasks. At present, most mainstream solutions establish the mapping between views and shape of an object by assembling the networks of 2D encoder and 3D decoder as the basic structure while they adopt different approaches to obtain aggregation of features from several views. Among them, the methods using attention-based fusion perform better and more stable than the others, however, they still have an obvious shortcoming – the strong independence of each view during predicting the weights for merging leads to a lack of adaption of the global state. In this paper, we propose a global-aware attention-based fusion approach that builds the correlation between each branch and the global to provide a comprehensive foundation for weights inference. In order to enhance the ability of the network, we introduce a novel loss function to supervise the shape overall and propose a dynamic two-stage training strategy that can effectively adapt to all reconstructors with attention-based fusion. Experiments on ShapeNet verify that our method outperforms existing SOTA methods while the amount of parameters is far less than the same type of algorithm, Pix2Vox++. Furthermore, we propose a view-reduction method based on maximizing diversity and discuss the cost-performance tradeoff of our model to achieve a better performance when facing heavy input amount and limited computational cost.

READ FULL TEXT
research
03/14/2022

VPFusion: Joint 3D Volume and Pixel-Aligned Feature Fusion for Single and Multi-view 3D Reconstruction

We introduce a unified single and multi-view neural implicit 3D reconstr...
research
08/20/2018

VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification

Multi-view deep neural network is perhaps the most successful approach i...
research
11/09/2021

PREMA: Part-based REcurrent Multi-view Aggregation Network for 3D Shape Retrieval

We propose the Part-based Recurrent Multi-view Aggregation network(PREMA...
research
03/24/2021

Multi-view 3D Reconstruction with Transformer

Deep CNN-based methods have so far achieved the state of the art results...
research
11/24/2019

Multi-View Time Series Classification via Global-Local Correlative Channel-Aware Fusion Mechanism

Multi-view time series classification aims to fuse the distinctive tempo...
research
05/17/2019

3DViewGraph: Learning Global Features for 3D Shapes from A Graph of Unordered Views with Attention

Learning global features by aggregating information over multiple views ...
research
10/04/2019

Higher Order Function Networks for View Planning and Multi-View Reconstruction

We consider the problem of planning views for a robot to acquire images ...

Please sign up or login with your details

Forgot password? Click here to reset