Category-Level 3D Non-Rigid Registration from Single-View RGB Images

08/17/2020
by   Diego Rodriguez, et al.
10

In this paper, we propose a novel approach to solve the 3D non-rigid registration problem from RGB images using Convolutional Neural Networks (CNNs). Our objective is to find a deformation field (typically used for transferring knowledge between instances, e.g., grasping skills) that warps a given 3D canonical model into a novel instance observed by a single-view RGB image. This is done by training a CNN that infers a deformation field for the visible parts of the canonical model and by employing a learned shape (latent) space for inferring the deformations of the occluded parts. As result of the registration, the observed model is reconstructed. Because our method does not need depth information, it can register objects that are typically hard to perceive with RGB-D sensors, e.g. with transparent or shiny surfaces. Even without depth data, our approach outperforms the Coherent Point Drift (CPD) registration method for the evaluated object categories.

READ FULL TEXT

page 1

page 3

page 4

page 5

page 6

page 7

research
09/14/2018

Transferring Category-based Functional Grasping Skills by Latent Space Non-Rigid Registration

Objects within a category are often similar in their shape and usage. Wh...
research
09/14/2018

Transferring Grasping Skills to Novel Instances by Latent Space Non-Rigid Registration

Robots acting in open environments need to be able to handle novel objec...
research
02/05/2018

3D non-rigid registration using color: Color Coherent Point Drift

Research into object deformations using computer vision techniques has b...
research
11/07/2021

Registration Techniques for Deformable Objects

In general, the problem of non-rigid registration is about matching two ...
research
03/21/2018

Non-rigid 3D Shape Registration using an Adaptive Template

We present a new fully-automatic non-rigid 3D shape registration (morphi...
research
07/07/2018

Representing a Partially Observed Non-Rigid 3D Human Using Eigen-Texture and Eigen-Deformation

Reconstruction of the shape and motion of humans from RGB-D is a challen...
research
01/14/2022

Multimodal registration of FISH and nanoSIMS images using convolutional neural network models

Nanoscale secondary ion mass spectrometry (nanoSIMS) and fluorescence in...

Please sign up or login with your details

Forgot password? Click here to reset