A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video

01/31/2018
by   Youngjae Yu, et al.
0

We address the problem of highlight detection from a 360 degree video by summarizing it both spatially and temporally. Given a long 360 degree video, we spatially select pleasantly-looking normal field-of-view (NFOV) segments from unlimited field of views (FOV) of the 360 degree video, and temporally summarize it into a concise and informative highlight as a selected subset of subshots. We propose a novel deep ranking model named as Composition View Score (CVS) model, which produces a spherical score map of composition per video segment, and determines which view is suitable for highlight via a sliding window kernel at inference. To evaluate the proposed framework, we perform experiments on the Pano2Vid benchmark dataset and our newly collected 360 degree video highlight dataset from YouTube and Vimeo. Through evaluation using both quantitative summarization metrics and user studies via Amazon Mechanical Turk, we demonstrate that our approach outperforms several state-of-the-art highlight detection methods. We also show that our model is 16 times faster at inference than AutoCam, which is one of the first summarization algorithms of 360 degree videos

READ FULL TEXT

page 1

page 3

page 4

page 7

page 8

research
05/08/2018

A Memory Network Approach for Story-based Temporal Summarization of 360° Videos

We address the problem of story-based temporal summarization of long 360...
research
03/03/2019

Less is More: Learning Highlight Detection from Video Duration

Highlight detection has the potential to significantly ease video browsi...
research
07/20/2020

MINI-Net: Multiple Instance Ranking Network for Video Highlight Detection

We address the weakly supervised video highlight detection problem for l...
research
06/24/2017

Large-Scale Mapping of Human Activity using Geo-Tagged Videos

This paper is the first work to perform spatio-temporal mapping of human...
research
01/30/2018

Object Detection in Videos by Short and Long Range Object Linking

We address the problem of detecting objects in videos with the interest ...
research
01/26/2019

Real-time Video Summarization on Commodity Hardware

We present a method for creating video summaries in real-time on commodi...
research
11/24/2017

For Your Eyes Only: Learning to Summarize First-Person Videos

With the increasing amount of video data, it is desirable to highlight o...

Please sign up or login with your details

Forgot password? Click here to reset