MonStereo: When Monocular and Stereo Meet at the Tail of 3D Human Localization

by   Lorenzo Bertoni, et al.

Monocular and stereo vision are cost-effective solutions for 3D human localization in the context of self-driving cars or social robots. However, they are usually developed independently and have their respective strengths and limitations. We propose a novel unified learning framework that leverages the strengths of both monocular and stereo cues for 3D human localization. Our method jointly (i) associates humans in left-right images, (ii) deals with occluded and distant cases in stereo settings by relying on the robustness of monocular cues, and (iii) tackles the intrinsic ambiguity of monocular perspective projection by exploiting prior knowledge of human height distribution. We achieve state-of-the-art quantitative results for the 3D localization task on KITTI dataset and estimate confidence intervals that account for challenging instances. We show qualitative examples for the long tail challenges such as occluded, far-away, and children instances.


page 2

page 7

page 9

page 10

page 11


MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation

We tackle the fundamentally ill-posed problem of 3D human localization f...

ChiTransformer:Towards Reliable Stereo from Cues

Current stereo matching techniques are challenged by restricted searchin...

SGM3D: Stereo Guided Monocular 3D Object Detection

Monocular 3D object detection is a critical yet challenging task for aut...

Perceiving Humans: from Monocular 3D Localization to Social Distancing

Perceiving humans in the context of Intelligent Transportation Systems (...

Triangulation Learning Network: from Monocular to Stereo 3D Object Detection

In this paper, we study the problem of 3D object detection from stereo i...

BirdSLAM: Monocular Multibody SLAM in Bird's-Eye View

In this paper, we present BirdSLAM, a novel simultaneous localization an...

Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modeling

The detection of small road hazards, such as lost cargo, is a vital capa...