InSeGAN: A Generative Approach to Segmenting Identical Instances in Depth Images

by   Anoop Cherian, et al.

In this paper, we present InSeGAN, an unsupervised 3D generative adversarial network (GAN) for segmenting (nearly) identical instances of rigid objects in depth images. Using an analysis-by-synthesis approach, we design a novel GAN architecture to synthesize a multiple-instance depth image with independent control over each instance. InSeGAN takes in a set of code vectors (e.g., random noise vectors), each encoding the 3D pose of an object that is represented by a learned implicit object template. The generator has two distinct modules. The first module, the instance feature generator, uses each encoded pose to transform the implicit template into a feature map representation of each object instance. The second module, the depth image renderer, aggregates all of the single-instance feature maps output by the first module and generates a multiple-instance depth image. A discriminator distinguishes the generated multiple-instance depth images from the distribution of true depth images. To use our model for instance segmentation, we propose an instance pose encoder that learns to take in a generated depth image and reproduce the pose code vectors for all of the object instances. To evaluate our approach, we introduce a new synthetic dataset, "Insta-10", consisting of 100,000 depth images, each with 5 instances of an object from one of 10 classes. Our experiments on Insta-10, as well as on real-world noisy depth images, show that InSeGAN achieves state-of-the-art performance, often outperforming prior methods by large margins.


page 1

page 7

page 11

page 12

page 13

page 16

page 17

page 18


TemplateNet for Depth-Based Object Instance Recognition

We present a novel deep architecture termed templateNet for depth based ...

Deep Watershed Transform for Instance Segmentation

Most contemporary approaches to instance segmentation use complex pipeli...

Uni6Dv2: Noise Elimination for 6D Pose Estimation

Few prior 6D pose estimation methods use a backbone network to extract f...

Unknown Object Segmentation from Stereo Images

Although instance-aware perception is a key prerequisite for many autono...

Learning Segmentation Masks with the Independence Prior

An instance with a bad mask might make a composite image that uses it lo...

Learning Rich Features from RGB-D Images for Object Detection and Segmentation

In this paper we study the problem of object detection for RGB-D images ...

CEN : Cooperatively Evolving Networks

A finitely repeated game is a dynamic game in which a simultaneous game ...