DeepAI AI Chat
Log In Sign Up

From a Bird's Eye View to See: Joint Camera and Subject Registration without the Camera Calibration

by   Zekun Qian, et al.
University of South Carolina
Tianjin University

We tackle a new problem of multi-view camera and subject registration in the bird's eye view (BEV) without pre-given camera calibration. This is a very challenging problem since its only input is several RGB images from different first-person views (FPVs) for a multi-person scene, without the BEV image and the calibration of the FPVs, while the output is a unified plane with the localization and orientation of both the subjects and cameras in a BEV. We propose an end-to-end framework solving this problem, whose main idea can be divided into following parts: i) creating a view-transform subject detection module to transform the FPV to a virtual BEV including localization and orientation of each pedestrian, ii) deriving a geometric transformation based method to estimate camera localization and view direction, i.e., the camera registration in a unified BEV, iii) making use of spatial and appearance information to aggregate the subjects into the unified BEV. We collect a new large-scale synthetic dataset with rich annotations for evaluation. The experimental results show the remarkable effectiveness of our proposed method.


page 3

page 8


Multiple Human Association between Top and Horizontal Views by Matching Subjects' Spatial Distributions

Video surveillance can be significantly enhanced by using both top-view ...

BEV-Locator: An End-to-end Visual Semantic Localization Network Using Multi-View Images

Accurate localization ability is fundamental in autonomous driving. Trad...

The WILDTRACK Multi-Camera Person Dataset

People detection methods are highly sensitive to the perpetual occlusion...

Learning Precise 3D Manipulation from Multiple Uncalibrated Cameras

In this work, we present an effective multi-view approach to closed-loop...

Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation

Accurately estimating the orientation of pedestrians is an important and...

Capturing Dynamic Textured Surfaces of Moving Targets

We present an end-to-end system for reconstructing complete watertight a...

Volumetric Capture of Humans with a Single RGBD Camera via Semi-Parametric Learning

Volumetric (4D) performance capture is fundamental for AR/VR content gen...