Localization and Mapping using Instance-specific Mesh Models

by   Qiaojun Feng, et al.

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.


page 1

page 6


Multi-Category Mesh Reconstruction From Image Collections

Recently, learning frameworks have shown the capability of inferring the...

Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images

We study the problem of shape generation in 3D mesh representation from ...

Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation

We study the problem of shape generation in 3D mesh representation from ...

Implicit Object Mapping With Noisy Data

Modelling individual objects as Neural Radiance Fields (NeRFs) within a ...

MO-LTR: Multiple Object Localization, Tracking, and Reconstruction from Monocular RGB Videos

Semantic aware reconstruction is more advantageous than geometric-only r...

Weak Multi-View Supervision for Surface Mapping Estimation

We propose a weakly-supervised multi-view learning approach to learn cat...

ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description

Autonomous systems need to understand the semantics and geometry of thei...