Localization and Mapping using Instance-specific Mesh Models

03/08/2021
by   Qiaojun Feng, et al.
0

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. This is an important problem because robots need rich understanding of geometry and context if they are to shape the future of transportation, construction, and agriculture. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images. Multi-view constraints on the object shape are obtained by detecting objects and extracting category-specific keypoints and segmentation masks. We show that the errors between projections of the mesh model and the observed keypoints and masks can be differentiated in order to obtain accurate instance-specific object shapes. We evaluate the performance of the proposed approach in simulation and on the KITTI dataset by building maps of car poses and shapes.

READ FULL TEXT

page 1

page 6

10/21/2021

Multi-Category Mesh Reconstruction From Image Collections

Recently, learning frameworks have shown the capability of inferring the...
04/21/2022

Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images

We study the problem of shape generation in 3D mesh representation from ...
08/05/2019

Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation

We study the problem of shape generation in 3D mesh representation from ...
04/22/2022

Implicit Object Mapping With Noisy Data

Modelling individual objects as Neural Radiance Fields (NeRFs) within a ...
12/09/2020

MO-LTR: Multiple Object Localization, Tracking, and Reconstruction from Monocular RGB Videos

Semantic aware reconstruction is more advantageous than geometric-only r...
05/04/2021

Weak Multi-View Supervision for Surface Mapping Estimation

We propose a weakly-supervised multi-view learning approach to learn cat...
08/01/2021

ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description

Autonomous systems need to understand the semantics and geometry of thei...