Optimal and Robust Category-level Perception: Object Pose and Shape Estimation from 2D and 3D Semantic Keypoints

06/24/2022
by   Jingnan Shi, et al.
7

We consider a category-level perception problem, where one is given 2D or 3D sensor data picturing an object of a given category (e.g., a car), and has to reconstruct the 3D pose and shape of the object despite intra-class variability (i.e., different car models have different shapes). We consider an active shape model, where – for an object category – we are given a library of potential CAD models describing objects in that category, and we adopt a standard formulation where pose and shape are estimated from 2D or 3D keypoints via non-convex optimization. Our first contribution is to develop PACE3D* and PACE2D*, the first certifiably optimal solvers for pose and shape estimation using 3D and 2D keypoints, respectively. Both solvers rely on the design of tight (i.e., exact) semidefinite relaxations. Our second contribution is to develop outlier-robust versions of both solvers, named PACE3D# and PACE2D#. Towards this goal, we propose ROBIN, a general graph-theoretic framework to prune outliers, which uses compatibility hypergraphs to model measurements' compatibility. We show that in category-level perception problems these hypergraphs can be built from winding orders of the keypoints (in 2D) or their convex hulls (in 3D), and many outliers can be pruned via maximum hyperclique computation. The last contribution is an extensive experimental evaluation. Besides providing an ablation study on simulated datasets and on the PASCAL dataset, we combine our solver with a deep keypoint detector, and show that PACE3D# improves over the state of the art in vehicle pose estimation in the ApolloScape datasets, and its runtime is compatible with practical applications.

READ FULL TEXT

page 1

page 7

page 9

page 15

page 30

page 31

page 33

page 34

research
04/16/2021

Optimal Pose and Shape Estimation for Category-level 3D Object Perception

We consider a category-level perception problem, where one is given 3D s...
research
09/07/2021

Certifiable Outlier-Robust Geometric Perception: Exact Semidefinite Relaxations and Scalable Global Optimization

We propose the first general and scalable framework to design certifiabl...
research
09/18/2019

Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection

Semidefinite Programming (SDP) and Sums-of-Squares (SOS) relaxations hav...
research
11/07/2020

ROBIN: a Graph-Theoretic Approach to Reject Outliers in Robust Estimation using Invariants

Many estimation problems in robotics, computer vision, and learning requ...
research
03/15/2019

kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation

We would like robots to achieve purposeful manipulation by placing any i...
research
11/27/2019

In Perfect Shape: Certifiably Optimal 3D Shape Reconstruction from 2D Landmarks

We study the problem of 3D shape reconstruction from 2D landmarks extrac...
research
03/04/2017

Sparse Depth Sensing for Resource-Constrained Robots

We consider the case in which a robot has to navigate in an unknown envi...

Please sign up or login with your details

Forgot password? Click here to reset