Particle Videos Revisited: Tracking Through Occlusions Using Point Trajectories

04/08/2022
by   Adam W. Harley, et al.
0

Tracking pixels in videos is typically studied as an optical flow estimation problem, where every pixel is described with a displacement vector that locates it in the next frame. Even though wider temporal context is freely available, prior efforts to take this into account have yielded only small gains over 2-frame methods. In this paper, we revisit Sand and Teller's "particle video" approach, and study pixel tracking as a long-range motion estimation problem, where every pixel is described with a trajectory that locates it in multiple future frames. We re-build this classic approach using components that drive the current state-of-the-art in flow and object tracking, such as dense cost maps, iterative optimization, and learned appearance updates. We train our models using long-range amodal point trajectories mined from existing optical flow datasets that we synthetically augment with occlusions. We test our approach in trajectory estimation benchmarks and in keypoint label propagation tasks, and compare favorably against state-of-the-art optical flow and feature tracking methods.

READ FULL TEXT

page 1

page 8

page 9

page 12

research
06/08/2023

Tracking Everything Everywhere All at Once

We present a new test-time optimization method for estimating dense and ...
research
07/14/2023

CoTracker: It is Better to Track Together

Methods for video motion prediction either estimate jointly the instanta...
research
04/02/2019

Multigrid Predictive Filter Flow for Unsupervised Learning on Videos

We introduce multigrid Predictive Filter Flow (mgPFF), a framework for u...
research
06/03/2023

Context-TAP: Tracking Any Point Demands Spatial Context Features

We tackle the problem of Tracking Any Point (TAP) in videos, which speci...
research
10/09/2020

Robust Instance Tracking via Uncertainty Flow

Current state-of-the-art trackers often fail due to distractorsand large...
research
05/22/2023

MFT: Long-Term Tracking of Every Pixel

We propose MFT – Multi-Flow dense Tracker – a novel method for dense, pi...
research
07/24/2023

Fast Full-frame Video Stabilization with Iterative Optimization

Video stabilization refers to the problem of transforming a shaky video ...

Please sign up or login with your details

Forgot password? Click here to reset