Attention-based 3D Object Reconstruction from a Single Image

08/11/2020
by   Andrey Salvi, et al.
0

Recently, learning-based approaches for 3D reconstruction from 2D images have gained popularity due to its modern applications, e.g., 3D printers, autonomous robots, self-driving cars, virtual reality, and augmented reality. The computer vision community has applied a great effort in developing functions to reconstruct the full 3D geometry of objects and scenes. However, to extract image features, they rely on convolutional neural networks, which are ineffective in capturing long-range dependencies. In this paper, we propose to substantially improve Occupancy Networks, a state-of-the-art method for 3D object reconstruction. For such we apply the concept of self-attention within the network's encoder in order to leverage complementary input features rather than those based on local regions, helping the encoder to extract global information. With our approach, we were capable of improving the original work in 5.05 Chamfer-L1 distance. We also perform a qualitative study that shows that our approach was able to generate much more consistent meshes, confirming its increased generalization power over the current state-of-the-art.

READ FULL TEXT

page 4

page 7

01/07/2021

Self-Attention Based Context-Aware 3D Object Detection

Most existing point-cloud based 3D object detectors use convolution-like...
02/17/2019

Exploring Stereovision-Based 3-D Scene Reconstruction for Augmented Reality

Three-dimensional (3-D) scene reconstruction is one of the key technique...
10/24/2018

Modeling Localness for Self-Attention Networks

Self-attention networks have proven to be of profound value for its stre...
10/12/2016

Image Based Camera Localization: an Overview

Recently, virtual reality, augmented reality, robotics, self-driving car...
09/02/2021

Studying the Effects of Self-Attention for Medical Image Analysis

When the trained physician interprets medical images, they understand th...
10/30/2020

An Unsupervised Approach towards Varying Human Skin Tone Using Generative Adversarial Networks

With the increasing popularity of augmented and virtual reality, retaile...
03/03/2019

X-Section: Cross-section Prediction for Enhanced RGBD Fusion

Detailed 3D reconstruction is an important challenge with application to...