Refining 6D Object Pose Predictions using Abstract Render-and-Compare

by   Arul Selvam Periyasamy, et al.

Robotic systems often require precise scene analysis capabilities, especially in unstructured, cluttered situations, as occurring in human-made environments. While current deep-learning based methods yield good estimates of object poses, they often struggle with large amounts of occlusion and do not take inter-object effects into account. Vision as inverse graphics is a promising concept for detailed scene analysis. A key element for this idea is a method for inferring scene parameter updates from the rasterized 2D scene. However, the rasterization process is notoriously difficult to invert, both due to the projection and occlusion process, but also due to secondary effects such as lighting or reflections. We propose to remove the latter from the process by mapping the rasterized image into an abstract feature space learned in a self-supervised way from pixel correspondences. Using only a light-weight inverse rendering module, this allows us to refine 6D object pose estimations in highly cluttered scenes by optimizing a simple pixel-wise difference in the abstract image representation. We evaluate our approach on the challenging YCB-Video dataset, where it yields large improvements and demonstrates a large basin of attraction towards the correct object poses.



There are no comments yet.


page 1

page 2

page 4

page 5

page 6


3DP3: 3D Scene Perception via Probabilistic Programming

We present 3DP3, a framework for inverse graphics that uses inference in...

Semi-Supervised Learning of Multi-Object 3D Scene Representations

Representing scenes at the granularity of objects is a prerequisite for ...

Neural Inverse Rendering of an Indoor Scene from a Single Image

Inverse rendering aims to estimate physical scene attributes (e.g., refl...

Table-Top Scene Analysis Using Knowledge-Supervised MCMC

In this paper, we propose a probabilistic method to generate abstract sc...

Pix2Pose: Pixel-Wise Coordinate Regression of Objects for 6D Pose Estimation

Estimating the 6D pose of objects using only RGB images remains challeng...

SporeAgent: Reinforced Scene-level Plausibility for Object Pose Refinement

Observational noise, inaccurate segmentation and ambiguity due to symmet...

PX-NET: Simple, Efficient Pixel-Wise Training of Photometric Stereo Networks

Retrieving accurate 3D reconstructions of objects from the way they refl...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.