DeepAI AI Chat
Log In Sign Up

Efficient Video Summarization Framework using EEG and Eye-tracking Signals

by   Sai Sukruth Bezugam, et al.

This paper proposes an efficient video summarization framework that will give a gist of the entire video in a few key-frames or video skims. Existing video summarization frameworks are based on algorithms that utilize computer vision low-level feature extraction or high-level domain level extraction. However, being the ultimate user of the summarized video, humans remain the most neglected aspect. Therefore, the proposed paper considers human's role in summarization and introduces human visual attention-based summarization techniques. To understand human attention behavior, we have designed and performed experiments with human participants using electroencephalogram (EEG) and eye-tracking technology. The EEG and eye-tracking data obtained from the experimentation are processed simultaneously and used to segment frames containing useful information from a considerable video volume. Thus, the frame segmentation primarily relies on the cognitive judgments of human beings. Using our approach, a video is summarized by 96.5 and high recall factors. The comparison with the state-of-the-art techniques demonstrates that the proposed approach yields ceiling-level performance with reduced computational cost in summarising the videos.


page 1

page 3

page 4

page 5

page 7

page 8

page 10


Use of Affective Visual Information for Summarization of Human-Centric Videos

Increasing volume of user-generated human-centric video content and thei...

The eyes know it: FakeET – An Eye-tracking Database to Understand Deepfake Perception

We present FakeET– an eye-tracking database to understand human visual p...

Video Summarization through Human Detection on a Social Robot

In this paper, we propose a novel video summarization system which captu...

A Deep Learning Approach for the Segmentation of Electroencephalography Data in Eye Tracking Applications

The collection of eye gaze information provides a window into many criti...

Key Frame Extraction with Attention Based Deep Neural Networks

Automatic keyframe detection from videos is an exercise in selecting sce...

Query-based Video Summarization with Pseudo Label Supervision

Existing datasets for manually labelled query-based video summarization ...

A new approach for a safe car assistance system

Drowsiness, which is the state when drivers do not have scheduled breaks...