RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

03/26/2020

∙

by   Zachary Teed, et al.

∙

3

∙

share

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network architecture for optical flow. RAFT extracts per-pixel features, builds multi-scale 4D correlation volumes for all pairs of pixels, and iteratively updates a flow field through a recurrent unit that performs lookups on the correlation volumes. RAFT achieves state-of-the-art performance, with strong cross-dataset generalization and high efficiency in inference time, training speed, and parameter count. Code is available <https://github.com/princeton-vl/RAFT>.