Synthetic Defocus and Look-Ahead Autofocus for Casual Videography

05/15/2019
by   Xuaner, et al.
7

In cinema, large camera lenses create beautiful shallow depth of field (DOF), but make focusing difficult and expensive. Accurate cinema focus usually relies on a script and a person to control focus in realtime. Casual videographers often crave cinematic focus, but fail to achieve it. We either sacrifice shallow DOF, as in smartphone videos; or we struggle to deliver accurate focus, as in videos from larger cameras. This paper is about a new approach in the pursuit of cinematic focus for casual videography. We present a system that synthetically renders refocusable video from a deep DOF video shot with a smartphone, and analyzes future video frames to deliver context-aware autofocus for the current frame. To create refocusable video, we extend recent machine learning methods designed for still photography, contributing a new dataset for machine training, a rendering model better suited to cinema focus, and a filtering solution for temporal coherence. To choose focus accurately for each frame, we demonstrate autofocus that looks at upcoming video frames and applies AI-assist modules such as motion, face, audio and saliency detection. We also show that autofocus benefits from machine learning and a large-scale video dataset with focus annotation, where we use our RVR-LAAF GUI to create this sizable dataset efficiently. We deliver, for example, a shallow DOF video where the autofocus transitions onto each person before she begins to speak. This is impossible for conventional camera autofocus because it would require seeing into the future.

READ FULL TEXT

page 1

page 5

page 6

page 7

page 8

page 13

page 14

research
11/30/2017

Future Person Localization in First-Person Videos

We present a new task that predicts future locations of people observed ...
research
04/26/2016

EgoSampling: Wide View Hyperlapse from Egocentric Videos

The possibility of sharing one's point of view makes use of wearable cam...
research
03/11/2016

Learning Gaze Transitions from Depth to Improve Video Saliency Estimation

In this paper we introduce a novel Depth-Aware Video Saliency approach t...
research
01/10/2019

Cricket stroke extraction: Towards creation of a large-scale cricket actions dataset

In this paper, we deal with the problem of temporal action localization ...
research
01/13/2022

MetaDance: Few-shot Dancing Video Retargeting via Temporal-aware Meta-learning

Dancing video retargeting aims to synthesize a video that transfers the ...
research
04/12/2022

Unidirectional Video Denoising by Mimicking Backward Recurrent Modules with Look-ahead Forward Ones

While significant progress has been made in deep video denoising, it rem...

Please sign up or login with your details

Forgot password? Click here to reset