InSeGAN: A Generative Approach to Segmenting Identical Instances in Depth Images

08/31/2021
by   Anoop Cherian, et al.
10

In this paper, we present InSeGAN, an unsupervised 3D generative adversarial network (GAN) for segmenting (nearly) identical instances of rigid objects in depth images. Using an analysis-by-synthesis approach, we design a novel GAN architecture to synthesize a multiple-instance depth image with independent control over each instance. InSeGAN takes in a set of code vectors (e.g., random noise vectors), each encoding the 3D pose of an object that is represented by a learned implicit object template. The generator has two distinct modules. The first module, the instance feature generator, uses each encoded pose to transform the implicit template into a feature map representation of each object instance. The second module, the depth image renderer, aggregates all of the single-instance feature maps output by the first module and generates a multiple-instance depth image. A discriminator distinguishes the generated multiple-instance depth images from the distribution of true depth images. To use our model for instance segmentation, we propose an instance pose encoder that learns to take in a generated depth image and reproduce the pose code vectors for all of the object instances. To evaluate our approach, we introduce a new synthetic dataset, "Insta-10", consisting of 100,000 depth images, each with 5 instances of an object from one of 10 classes. Our experiments on Insta-10, as well as on real-world noisy depth images, show that InSeGAN achieves state-of-the-art performance, often outperforming prior methods by large margins.

READ FULL TEXT

page 1

page 7

page 11

page 12

page 13

page 16

page 17

page 18

11/10/2015

TemplateNet for Depth-Based Object Instance Recognition

We present a novel deep architecture termed templateNet for depth based ...
11/24/2016

Deep Watershed Transform for Instance Segmentation

Most contemporary approaches to instance segmentation use complex pipeli...
08/15/2022

Uni6Dv2: Noise Elimination for 6D Pose Estimation

Few prior 6D pose estimation methods use a backbone network to extract f...
03/11/2021

Unknown Object Segmentation from Stereo Images

Although instance-aware perception is a key prerequisite for many autono...
11/12/2018

Learning Segmentation Masks with the Independence Prior

An instance with a bad mask might make a composite image that uses it lo...
07/22/2014

Learning Rich Features from RGB-D Images for Object Detection and Segmentation

In this paper we study the problem of object detection for RGB-D images ...
07/05/2022

CEN : Cooperatively Evolving Networks

A finitely repeated game is a dynamic game in which a simultaneous game ...