Deep 360^∘ Optical Flow Estimation Based on Multi-Projection Fusion

07/27/2022
by   Yiheng Li, et al.
5

Optical flow computation is essential in the early stages of the video processing pipeline. This paper focuses on a less explored problem in this area, the 360^∘ optical flow estimation using deep neural networks to support increasingly popular VR applications. To address the distortions of panoramic representations when applying convolutional neural networks, we propose a novel multi-projection fusion framework that fuses the optical flow predicted by the models trained using different projection methods. It learns to combine the complementary information in the optical flow results under different projections. We also build the first large-scale panoramic optical flow dataset to support the training of neural networks and the evaluation of panoramic optical flow estimation methods. The experimental results on our dataset demonstrate that our method outperforms the existing methods and other alternative deep networks that were developed for processing 360 content.

READ FULL TEXT

page 8

page 10

page 12

page 14

research
10/24/2019

Learning Multi-Human Optical Flow

The optical flow of humans is well known to be useful for the analysis o...
research
12/05/2016

Multi-way Particle Swarm Fusion

This paper proposes a novel MAP inference framework for Markov Random Fi...
research
07/11/2023

Image-Processing Based Methods to Improve the Robustness of Robotic Gripping

Image processing techniques have huge impact on most fields of robotics ...
research
01/29/2018

Hierarchical Spatial Transformer Network

Computer vision researchers have been expecting that neural networks hav...
research
11/20/2021

CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation

In this paper, we study the problem of jointly estimating the optical fl...
research
10/15/2020

Revisiting Optical Flow Estimation in 360 Videos

Nowadays 360 video analysis has become a significant research topic in t...
research
10/28/2020

GloFlow: Global Image Alignment for Creation of Whole Slide Images for Pathology from Video

The application of deep learning to pathology assumes the existence of d...

Please sign up or login with your details

Forgot password? Click here to reset