Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks

01/26/2019
by   Stefan Milz, et al.
1

We present the first approach for 3D point-cloud to image translation based on conditional Generative Adversarial Networks (cGAN). The model handles multi-modal information sources from different domains, i.e. raw point-sets and images. The generator is capable of processing three conditions, whereas the point-cloud is encoded as raw point-set and camera projection. An image background patch is used as constraint to bias environmental texturing. A global approximation function within the generator is directly applied on the point-cloud (Point-Net). Hence, the representative learning model incorporates global 3D characteristics directly at the latent feature space. Conditions are used to bias the background and the viewpoint of the generated image. This opens up new ways in augmenting or texturing 3D data to aim the generation of fully individual images. We successfully evaluated our method on the Kitti and SunRGBD dataset with an outstanding object detection inception score.

READ FULL TEXT

page 3

page 4

page 6

page 7

page 8

research
10/13/2020

Procedural 3D Terrain Generation using Generative Adversarial Networks

Procedural 3D Terrain generation has become a necessity in open world ga...
research
11/24/2016

3D Fully Convolutional Network for Vehicle Detection in Point Cloud

2D fully convolutional network has been recently successfully applied to...
research
05/04/2020

Transforming and Projecting Images into Class-conditional Generative Networks

We present a method for projecting an input image into the space of a cl...
research
02/02/2020

Adversarial Generation of Continuous Implicit Shape Representations

This work presents a generative adversarial architecture for generating ...
research
09/14/2021

Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Images

In this work, we present a simple yet effective framework to address th...
research
01/21/2018

PU-Net: Point Cloud Upsampling Network

Learning and analyzing 3D point cloud with deep networks is challenging ...
research
09/02/2021

4D-Net for Learned Multi-Modal Alignment

We present 4D-Net, a 3D object detection approach, which utilizes 3D Poi...

Please sign up or login with your details

Forgot password? Click here to reset