DeepAI AI Chat
Log In Sign Up

DUT: Learning Video Stabilization by Simply Watching Unstable Videos

by   Yufei Xu, et al.

We propose a Deep Unsupervised Trajectory-based stabilization framework (DUT) in this paper. Traditional stabilizers focus on trajectory-based smoothing, which is controllable but fragile in occluded and textureless cases regarding the usage of hand-crafted features. On the other hand, previous deep video stabilizers directly generate stable videos in a supervised manner without explicit trajectory estimation, which is robust but less controllable and the appropriate paired data are hard to obtain. To construct a controllable and robust stabilizer, DUT makes the first attempt to stabilize unstable videos by explicitly estimating and smoothing trajectories in an unsupervised deep learning manner, which is composed of a DNN-based keypoint detector and motion estimator to generate grid-based trajectories, and a DNN-based trajectory smoother to stabilize videos. We exploit both the nature of continuity in motion and the consistency of keypoints and grid vertices before and after stabilization for unsupervised training. Experiment results on public benchmarks show that DUT outperforms representative state-of-the-art methods both qualitatively and quantitatively.


page 1

page 7

page 8

page 12

page 13

page 16


Articulated motion discovery using pairs of trajectories

We propose an unsupervised approach for discovering characteristic motio...

Human Trajectory Prediction using Spatially aware Deep Attention Models

Trajectory Prediction of dynamic objects is a widely studied topic in th...

AutoTrajectory: Label-free Trajectory Extraction and Prediction from Videos using Dynamic Points

Current methods for trajectory prediction operate in supervised manners,...

Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction

We propose a deep video prediction model conditioned on a single image a...

KEMP: Keyframe-Based Hierarchical End-to-End Deep Model for Long-Term Trajectory Prediction

Predicting future trajectories of road agents is a critical task for aut...

Hand-tremor frequency estimation in videos

We focus on the problem of estimating human hand-tremor frequency from i...

Deep Space-Time Video Upsampling Networks

Video super-resolution (VSR) and frame interpolation (FI) are traditiona...