Transfoming Multi-Concept Attention into Video Summarization

06/02/2020
by   Yen-Ting Liu*, et al.
0

Video summarization is among challenging tasks in computer vision, which aims at identifying highlight frames or shots over a lengthy video input. In this paper, we propose an novel attention-based framework for video summarization with complex video data. Unlike previous works which only apply attention mechanism on the correspondence between frames, our multi-concept video self-attention (MC-VSA) model is presented to identify informative regions across temporal and concept video features, which jointly exploit context diversity over time and space for summarization purposes. Together with consistency between video and summary enforced in our framework, our model can be applied to both labeled and unlabeled data, making our method preferable to real-world applications. Extensive and complete experiments on two benchmarks demonstrate the effectiveness of our model both quantitatively and qualitatively, and confirms its superiority over the stateof-the-arts.

READ FULL TEXT

page 2

page 10

page 12

research
06/02/2020

Transforming Multi-Concept Attention into Video Summarization

Video summarization is among challenging tasks in computer vision, which...
research
01/27/2022

Exploring Global Diversity and Local Context for Video Summarization

Video summarization aims to automatically generate a diverse and concise...
research
09/23/2020

Exploring global diverse attention via pairwise temporal relation for video summarization

Video summarization is an effective way to facilitate video searching an...
research
08/07/2017

Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowd-sourced Time-Sync Comments

With the prevalence of video sharing, there are increasing demands for a...
research
05/17/2021

AudioVisual Video Summarization

Audio and vision are two main modalities in video data. Multimodal learn...
research
05/13/2021

DeepQAMVS: Query-Aware Hierarchical Pointer Networks for Multi-Video Summarization

The recent growth of web video sharing platforms has increased the deman...
research
04/23/2021

Supervised Video Summarization via Multiple Feature Sets with Parallel Attention

The assignment of importance scores to particular frames or (short) segm...

Please sign up or login with your details

Forgot password? Click here to reset