DUT: Learning Video Stabilization by Simply Watching Unstable Videos

11/30/2020
by   Yufei Xu, et al.
0

We propose a Deep Unsupervised Trajectory-based stabilization framework (DUT) in this paper. Traditional stabilizers focus on trajectory-based smoothing, which is controllable but fragile in occluded and textureless cases regarding the usage of hand-crafted features. On the other hand, previous deep video stabilizers directly generate stable videos in a supervised manner without explicit trajectory estimation, which is robust but less controllable and the appropriate paired data are hard to obtain. To construct a controllable and robust stabilizer, DUT makes the first attempt to stabilize unstable videos by explicitly estimating and smoothing trajectories in an unsupervised deep learning manner, which is composed of a DNN-based keypoint detector and motion estimator to generate grid-based trajectories, and a DNN-based trajectory smoother to stabilize videos. We exploit both the nature of continuity in motion and the consistency of keypoints and grid vertices before and after stabilization for unsupervised training. Experiment results on public benchmarks show that DUT outperforms representative state-of-the-art methods both qualitatively and quantitatively.

READ FULL TEXT

page 1

page 7

page 8

page 12

page 13

page 16

research
11/28/2014

Articulated motion discovery using pairs of trajectories

We propose an unsupervised approach for discovering characteristic motio...
research
05/26/2017

Human Trajectory Prediction using Spatially aware Deep Attention Models

Trajectory Prediction of dynamic objects is a widely studied topic in th...
research
07/11/2020

AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points

Current methods for trajectory prediction operate in supervised manners,...
research
08/16/2023

DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory

Controllable video generation has gained significant attention in recent...
research
09/10/2018

Hand-tremor frequency estimation in videos

We focus on the problem of estimating human hand-tremor frequency from i...
research
10/01/2018

Unsupervised Trajectory Segmentation and Promoting of Multi-Modal Surgical Demonstrations

To improve the efficiency of surgical trajectory segmentation for robot ...
research
06/03/2023

Context-TAP: Tracking Any Point Demands Spatial Context Features

We tackle the problem of Tracking Any Point (TAP) in videos, which speci...

Please sign up or login with your details

Forgot password? Click here to reset