KFNet: Learning Temporal Camera Relocalization using Kalman Filtering

by   Lei Zhou, et al.
Tsinghua University
The Hong Kong University of Science and Technology

Temporal camera relocalization estimates the pose with respect to each video frame in sequence, as opposed to one-shot relocalization which focuses on a still image. Even though the time dependency has been taken into account, current temporal relocalization methods still generally underperform the state-of-the-art one-shot approaches in terms of accuracy. In this work, we improve the temporal relocalization method by using a network architecture that incorporates Kalman filtering (KFNet) for online camera relocalization. In particular, KFNet extends the scene coordinate regression problem to the time domain in order to recursively establish 2D and 3D correspondences for the pose determination. The network architecture design and the loss formulation are based on Kalman filtering in the context of Bayesian learning. Extensive experiments on multiple relocalization benchmarks demonstrate the high accuracy of KFNet at the top of both one-shot and temporal relocalization approaches. Our codes are released at https://github.com/zlthinker/KFNet.


page 5

page 7

page 12


Revisiting Temporal Modeling for Video-based Person ReID

Video-based person reID is an important task, which has received much at...

Few-Shot Video Object Detection

We introduce Few-Shot Video Object Detection (FSVOD) with three importan...

Full-Frame Scene Coordinate Regression for Image-Based Localization

Image-based localization, or camera relocalization, is a fundamental pro...

Visual Localization via Few-Shot Scene Region Classification

Visual (re)localization addresses the problem of estimating the 6-DoF (D...

A Code for Unscented Kalman Filtering on Manifolds (UKF-M)

The present paper introduces a novel methodology for Unscented Kalman Fi...

Automatic cinematography for 360 video

We describe our method for automatic generation of a visually interestin...

TransNet V2: An effective deep network architecture for fast shot transition detection

Although automatic shot transition detection approaches are already inve...

Code Repositories


KFNet: Learning Temporal Camera Relocalization using Kalman Filtering (CVPR 2020 Oral)

view repo

Please sign up or login with your details

Forgot password? Click here to reset