Fast Video Shot Transition Localization with Deep Structured Models

08/13/2018
by   Shitao Tang, et al.
0

Detection of video shot transition is a crucial pre-processing step in video analysis. Previous studies are restricted on detecting sudden content changes between frames through similarity measurement and multi-scale operations are widely utilized to deal with transitions of various lengths. However, localization of gradual transitions are still under-explored due to the high visual similarity between adjacent frames. Cut shot transitions are abrupt semantic breaks while gradual shot transitions contain low-level spatial-temporal patterns caused by video effects in addition to the gradual semantic breaks, e.g. dissolve. In order to address the problem, we propose a structured network which is able to detect these two shot transitions using targeted models separately. Considering speed performance trade-offs, we design a smart framework. With one TITAN GPU, the proposed method can achieve a 30× real-time speed. Experiments on public TRECVID07 and RAI databases show that our method outperforms the state-of-the-art methods. In order to train a high-performance shot transition detector, we contribute a new database ClipShots, which contains 128636 cut transitions and 38120 gradual transitions from 4039 online videos. ClipShots intentionally collect short videos for more hard cases caused by hand-held camera vibrations, large object motions, and occlusion.

READ FULL TEXT
05/09/2017

Large-scale, Fast and Accurate Shot Boundary Detection through Spatio-temporal Convolutional Neural Networks

Shot boundary detection (SBD) is an important pre-processing step for vi...
05/23/2017

Ridiculously Fast Shot Boundary Detection with Fully Convolutional Neural Networks

Shot boundary detection (SBD) is an important component of many video an...
08/11/2020

TransNet V2: An effective deep network architecture for fast shot transition detection

Although automatic shot transition detection approaches are already inve...
06/08/2019

TransNet: A deep network for fast detection of common shot transitions

Shot boundary detection (SBD) is an important first step in many video p...
11/11/2015

Hierarchical Recurrent Neural Encoder for Video Representation with Application to Captioning

Recently, deep learning approach, especially deep Convolutional Neural N...
07/27/2022

AutoTransition: Learning to Recommend Video Transition Effects

Video transition effects are widely used in video editing to connect sho...
04/12/2023

AutoShot: A Short Video Dataset and State-of-the-Art Shot Boundary Detection

The short-form videos have explosive popularity and have dominated the n...

Code Repositories

ClipShots

ClipShots is the first large-scale dataset for shot boundary detection collected from Youtube and Weibo covering more than 20 categories, including sports, TV shows, animals, etc.


view repo

Please sign up or login with your details

Forgot password? Click here to reset