DeepAI AI Chat
Log In Sign Up

6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-based Instance Representation Learning

by   Lu Zou, et al.

This paper presents 6D-ViT, a transformer-based instance representation learning network, which is suitable for highly accurate category-level object pose estimation on RGB-D images. Specifically, a novel two-stream encoder-decoder framework is dedicated to exploring complex and powerful instance representations from RGB images, point clouds and categorical shape priors. For this purpose, the whole framework consists of two main branches, named Pixelformer and Pointformer. The Pixelformer contains a pyramid transformer encoder with an all-MLP decoder to extract pixelwise appearance representations from RGB images, while the Pointformer relies on a cascaded transformer encoder and an all-MLP decoder to acquire the pointwise geometric characteristics from point clouds. Then, dense instance representations (i.e., correspondence matrix, deformation field) are obtained from a multi-source aggregation network with shape priors, appearance and geometric information as input. Finally, the instance 6D pose is computed by leveraging the correspondence among dense representations, shape priors, and the instance point clouds. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed 3D instance representation learning framework achieves state-of-the-art performance on both datasets, and significantly outperforms all existing methods.


page 1

page 4

page 5

page 8

page 9

page 10

page 12


Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networks

Category-level 6D pose estimation, aiming to predict the location and or...

Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation

We present a novel learning approach to recover the 6D poses and sizes o...

Generative Category-Level Shape and Pose Estimation with Semantic Primitives

Empowering autonomous agents with 3D understanding for daily objects is ...

Depth-based 6DoF Object Pose Estimation using Swin Transformer

Accurately estimating the 6D pose of objects is crucial for many applica...

SO(3)-Pose: SO(3)-Equivariance Learning for 6D Object Pose Estimation

6D pose estimation of rigid objects from RGB-D images is crucial for obj...

Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds

Dense captioning in 3D point clouds is an emerging vision-and-language t...

PersonalTailor: Personalizing 2D Pattern Design from 3D Garment Point Clouds

Garment pattern design aims to convert a 3D garment to the corresponding...