Log In Sign Up

For Your Eyes Only: Learning to Summarize First-Person Videos

by   Hsuan-I Ho, et al.

With the increasing amount of video data, it is desirable to highlight or summarize the videos of interest for viewing, search, or storage purposes. However, existing summarization approaches are typically trained from third-person videos, which cannot generalize to highlight the first-person ones. By advancing deep learning techniques, we propose a unique network architecture for transferring spatiotemporal information across video domains, which jointly solves metric-learning based feature embedding and keyframe selection via Bidirectional Long Short-Term Memory (BiLSTM). A practical semi-supervised learning setting is considered, i.e., only fully annotated third-person videos, unlabeled first-person videos, and a small amount of annotated first-person ones are required to train our proposed model. Qualitative and quantitative evaluations are performed in our experiments, which confirm that our model performs favorably against baseline and state-of-the-art approaches on first-person video summarization.


page 1

page 3

page 8


Video Summarization with Long Short-term Memory

We propose a novel supervised learning technique for summarizing videos ...

Making Third Person Techniques Recognize First-Person Actions in Egocentric Videos

We focus on first-person action recognition from egocentric videos. Unli...

Removing Rain in Videos: A Large-scale Database and A Two-stream ConvLSTM Approach

Rain removal has recently attracted increasing research attention, as it...

Temporally Coherent Bayesian Models for Entity Discovery in Videos by Tracklet Clustering

A video can be represented as a sequence of tracklets, each spanning 10-...

What I See Is What You See: Joint Attention Learning for First and Third Person Video Co-analysis

In recent years, more and more videos are captured from the first-person...

Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders

With the growing popularity of short-form video sharing platforms such a...

A Deep Ranking Model for Spatio-Temporal Highlight Detection from a 360 Video

We address the problem of highlight detection from a 360 degree video by...