For Your Eyes Only: Learning to Summarize First-Person Videos

11/24/2017
by   Hsuan-I Ho, et al.
0

With the increasing amount of video data, it is desirable to highlight or summarize the videos of interest for viewing, search, or storage purposes. However, existing summarization approaches are typically trained from third-person videos, which cannot generalize to highlight the first-person ones. By advancing deep learning techniques, we propose a unique network architecture for transferring spatiotemporal information across video domains, which jointly solves metric-learning based feature embedding and keyframe selection via Bidirectional Long Short-Term Memory (BiLSTM). A practical semi-supervised learning setting is considered, i.e., only fully annotated third-person videos, unlabeled first-person videos, and a small amount of annotated first-person ones are required to train our proposed model. Qualitative and quantitative evaluations are performed in our experiments, which confirm that our model performs favorably against baseline and state-of-the-art approaches on first-person video summarization.

READ FULL TEXT

page 1

page 3

page 8

research
05/26/2016

Video Summarization with Long Short-term Memory

We propose a novel supervised learning technique for summarizing videos ...
research
10/17/2019

Making Third Person Techniques Recognize First-Person Actions in Egocentric Videos

We focus on first-person action recognition from egocentric videos. Unli...
research
06/06/2019

Removing Rain in Videos: A Large-scale Database and A Two-stream ConvLSTM Approach

Rain removal has recently attracted increasing research attention, as it...
research
04/16/2019

What I See Is What You See: Joint Attention Learning for First and Third Person Video Co-analysis

In recent years, more and more videos are captured from the first-person...
research
09/22/2014

Temporally Coherent Bayesian Models for Entity Discovery in Videos by Tracklet Clustering

A video can be represented as a sequence of tracklets, each spanning 10-...
research
10/06/2015

Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders

With the growing popularity of short-form video sharing platforms such a...
research
01/31/2018

A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video

We address the problem of highlight detection from a 360 degree video by...

Please sign up or login with your details

Forgot password? Click here to reset