Blind Inpainting of Large-scale Masks of Thin Structures with Adversarial and Reinforcement Learning

12/05/2019
by   Hao Chen, et al.
13

Several imaging applications (vessels, retina, plant roots, road networks from satellites) require the accurate segmentation of thin structures for subsequent analysis. Discontinuities (gaps) in the extracted foreground may hinder down-stream image-based analysis of biomarkers, organ structure and topology. In this paper, we propose a general post-processing technique to recover such gaps in large-scale segmentation masks. We cast this problem as a blind inpainting task, where the regions of missing lines in the segmentation masks are not known to the algorithm, which we solve with an adversarially trained neural network. One challenge of using large images is the memory capacity of current GPUs. The typical approach of dividing a large image into smaller patches to train the network does not guarantee global coherence of the reconstructed image that preserves structure and topology. We use adversarial training and reinforcement learning (Policy Gradient) to endow the model with both global context and local details. We evaluate our method in several datasets in medical imaging, plant science, and remote sensing. Our experiments demonstrate that our model produces the most realistic and complete inpainted results, outperforming other approaches. In a dedicated study on plant roots we find that our approach is also comparable to human performance. Implementation available at <https://github.com/Hhhhhhhhhhao/Thin-Structure-Inpainting>.

READ FULL TEXT

page 1

page 2

page 4

page 5

page 8

page 10

page 12

research
07/12/2022

Shape-Aware Masking for Inpainting in Medical Imaging

Inpainting has recently been proposed as a successful deep learning tech...
research
03/15/2020

VCNet: A Robust Approach to Blind Image Inpainting

Blind inpainting is a task to automatically complete visual contents wit...
research
09/09/2019

Adversarial Policy Gradient for Deep Learning Image Augmentation

The use of semantic segmentation for masking and cropping input images h...
research
06/12/2023

Topology Repairing of Disconnected Pulmonary Airways and Vessels: Baselines and a Dataset

Accurate segmentation of pulmonary airways and vessels is crucial for th...
research
03/27/2023

Blind Inpainting with Object-aware Discrimination for Artificial Marker Removal

Medical images often contain artificial markers added by doctors, which ...
research
03/23/2023

Generative Image Inpainting with Segmentation Confusion Adversarial Training and Contrastive Learning

This paper presents a new adversarial training framework for image inpai...
research
09/15/2023

Segmentation of Tubular Structures Using Iterative Training with Tailored Samples

We propose a minimal path method to simultaneously compute segmentation ...

Please sign up or login with your details

Forgot password? Click here to reset