Sparse Pose Trajectory Completion

by   Bo Liu, et al.

We propose a method to learn, even using a dataset where objects appear only in sparsely sampled views (e.g. Pix3D), the ability to synthesize a pose trajectory for an arbitrary reference image. This is achieved with a cross-modal pose trajectory transfer mechanism. First, a domain transfer function is trained to predict, from an RGB image of the object, its 2D depth map. Then, a set of image views is generated by learning to simulate object rotation in the depth space. Finally, the generated poses are mapped from this latent space into a set of corresponding RGB images using a learned identity preserving transform. This results in a dense pose trajectory of the object in image space. For each object type (e.g., a specific Ikea chair model), a 3D CAD model is used to render a full pose trajectory of 2D depth maps. In the absence of dense pose sampling in image space, these latent space trajectories provide cross-modal guidance for learning. The learned pose trajectories can be transferred to unseen examples, effectively synthesizing all object views in image space. Our method is evaluated on the Pix3D and ShapeNet datasets, in the setting of novel view synthesis under sparse pose supervision, demonstrating substantial improvements over recent art.


Cross-modal Deep Variational Hand Pose Estimation

The human hand moves in complex and high-dimensional ways, making estima...

Multi-path Learning for Object Pose Estimation Across Domains

We introduce a scalable approach for object pose estimation trained on s...

RGBD-Net: Predicting color and depth images for novel views synthesis

We address the problem of novel view synthesis from an unstructured set ...

Implicit 3D Orientation Learning for 6D Object Detection from RGB Images

We propose a real-time RGB-based pipeline for object detection and 6D po...

SO(3)-Pose: SO(3)-Equivariance Learning for 6D Object Pose Estimation

6D pose estimation of rigid objects from RGB-D images is crucial for obj...

Estimating Small Differences in Car-Pose from Orbits

Distinction among nearby poses and among symmetries of an object is chal...

LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments

We present LaLaLoc to localise in environments without the need for prio...

Please sign up or login with your details

Forgot password? Click here to reset