DeepAI AI Chat
Log In Sign Up

Video Compressive Sensing for Spatial Multiplexing Cameras using Motion-Flow Models

by   Aswin C. Sankaranarayanan, et al.
Carnegie Mellon University
cornell university
Rice University

Spatial multiplexing cameras (SMCs) acquire a (typically static) scene through a series of coded projections using a spatial light modulator (e.g., a digital micro-mirror device) and a few optical sensors. This approach finds use in imaging applications where full-frame sensors are either too expensive (e.g., for short-wave infrared wavelengths) or unavailable. Existing SMC systems reconstruct static scenes using techniques from compressive sensing (CS). For videos, however, existing acquisition and recovery methods deliver poor quality. In this paper, we propose the CS multi-scale video (CS-MUVI) sensing and recovery framework for high-quality video acquisition and recovery using SMCs. Our framework features novel sensing matrices that enable the efficient computation of a low-resolution video preview, while enabling high-resolution video recovery using convex optimization. To further improve the quality of the reconstructed videos, we extract optical-flow estimates from the low-resolution previews and impose them as constraints in the recovery procedure. We demonstrate the efficacy of our CS-MUVI framework for a host of synthetic and real measured SMC video data, and we show that high-quality videos can be recovered at roughly 60× compression.


page 2

page 6

page 9

page 11

page 12

page 13

page 14

page 19


Compressive Coded Aperture Keyed Exposure Imaging with Optical Flow Reconstruction

This paper describes a coded aperture and keyed exposure approach to com...

Compressive Acquisition of Dynamic Scenes

Compressive sensing (CS) is a new approach for the acquisition and recov...

CSVideoNet: A Real-time End-to-end Learning Framework for High-frame-rate Video Compressive Sensing

This paper addresses the real-time encoding-decoding problem for high-fr...

Stitched Panoramas from Toy Airborne Video Cameras

Effective panoramic photographs are taken from vantage points that are h...

FPA-CS: Focal Plane Array-based Compressive Imaging in Short-wave Infrared

Cameras for imaging in short and mid-wave infrared spectra are significa...

LiSens --- A Scalable Architecture for Video Compressive Sensing

The measurement rate of cameras that take spatially multiplexed measurem...

Dual-view Snapshot Compressive Imaging via Optical Flow Aided Recurrent Neural Network

Dual-view snapshot compressive imaging (SCI) aims to capture videos from...