Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images

11/26/2020
by   Qing Zhu, et al.
20

Detecting the changes of buildings in urban environments is essential. Existing methods that use only nadir images suffer from severe problems of ambiguous features and occlusions between buildings and other regions. Furthermore, buildings in urban environments vary significantly in scale, which leads to performance issues when using single-scale features. To solve these issues, this paper proposes a fused feature pyramid network, which utilizes both color and depth data for the 3D verification of existing buildings 2D footprints from oblique images. First, the color data of oblique images are enriched with the depth information rendered from 3D mesh models. Second, multiscale features are fused in the feature pyramid network to convolve both the color and depth data. Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings. Experimental evaluations using both the ISPRS benchmark datasets and Shenzhen datasets reveal that the proposed method outperforms the ResNet and EfficientNet networks by 5% and 2%, respectively, in terms of recall rate and precision. We demonstrate that the proposed method can successfully detect all changed buildings; therefore, only those marked as changed need to be manually checked during the pipeline updating procedure; this significantly reduces the manual quality control requirements. Moreover, ablation studies indicate that using depth data, feature pyramid modules, and multi-view voting strategies can lead to clear and progressive improvements.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 3

page 7

page 9

page 11

page 16

page 17

12/18/2021

3D Instance Segmentation of MVS Buildings

We present a novel framework for instance segmentation of 3D buildings f...
10/17/2020

MeshMVS: Multi-View Stereo Guided Mesh Reconstruction

Deep learning based 3D shape generation methods generally utilize latent...
02/21/2020

Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction

Integration of aerial and ground images has been proved as an efficient ...
08/01/2018

Attention-based Pyramid Aggregation Network for Visual Place Recognition

Visual place recognition is challenging in the urban environment and is ...
12/18/2019

Cost Volume Pyramid Based Depth Inference for Multi-View Stereo

We propose a cost volume based neural network for depth inference from m...
05/11/2021

EDPN: Enhanced Deep Pyramid Network for Blurry Image Restoration

Image deblurring has seen a great improvement with the development of de...
11/04/2021

Tea Chrysanthemum Detection under Unstructured Environments Using the TC-YOLO Model

Tea chrysanthemum detection at its flowering stage is one of the key com...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.