Dual-view Snapshot Compressive Imaging via Optical Flow Aided Recurrent Neural Network

09/11/2021
by   Ruiying Lu, et al.
4

Dual-view snapshot compressive imaging (SCI) aims to capture videos from two field-of-views (FoVs) using a 2D sensor (detector) in a single snapshot, achieving joint FoV and temporal compressive sensing, and thus enjoying the advantages of low-bandwidth, low-power, and low-cost. However, it is challenging for existing model-based decoding algorithms to reconstruct each individual scene, which usually require exhaustive parameter tuning with extremely long running time for large scale data. In this paper, we propose an optical flow-aided recurrent neural network for dual video SCI systems, which provides high-quality decoding in seconds. Firstly, we develop a diversity amplification method to enlarge the differences between scenes of two FoVs, and design a deep convolutional neural network with dual branches to separate different scenes from the single measurement. Secondly, we integrate the bidirectional optical flow extracted from adjacent frames with the recurrent neural network to jointly reconstruct each video in a sequential manner. Extensive results on both simulation and real data demonstrate the superior performance of our proposed model in a short inference time. The code and data are available at https://github.com/RuiyingLu/OFaNet-for-Dual-view-SCI.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 7

page 14

page 19

page 20

page 21

page 22

03/30/2020

Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging

Snapshot compressive imaging (SCI) aims to capture the high-dimensional ...
06/26/2013

Compressive Coded Aperture Keyed Exposure Imaging with Optical Flow Reconstruction

This paper describes a coded aperture and keyed exposure approach to com...
03/09/2015

Video Compressive Sensing for Spatial Multiplexing Cameras using Motion-Flow Models

Spatial multiplexing cameras (SMCs) acquire a (typically static) scene t...
03/26/2020

RAFT: Recurrent All-Pairs Field Transforms for Optical Flow

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep net...
10/13/2021

Optical-Flow-Reuse-Based Bidirectional Recurrent Network for Space-Time Video Super-Resolution

In this paper, we consider the task of space-time video super-resolution...
06/01/2020

High-quality Panorama Stitching based on Asymmetric Bidirectional Optical Flow

In this paper, we propose a panorama stitching algorithm based on asymme...
06/28/2019

Robustness Guarantees for Deep Neural Networks on Videos

The widespread adoption of deep learning models places demands on their ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.