Synchronous Prediction of Arousal and Valence Using LSTM Network for Affective Video Content Analysis

06/01/2018
by   Ligang Zhang, et al.
0

The affect embedded in video data conveys high-level semantic information about the content and has direct impact on the understanding and perception of reviewers, as well as their emotional responses. Affective Video Content Analysis (AVCA) attempts to generate a direct mapping between video content and the corresponding affective states such as arousal and valence dimensions. Most existing studies establish the mapping for each dimension separately using knowledge-based rules or traditional classifiers such as Support Vector Machine (SVM). The inherent correlations between affective dimensions have largely been unexploited, which are anticipated to include important information for accurate prediction of affective dimensions. To address this issue, this paper presents an approach to predict arousal and valance dimensions synchronously using the Long Short Term Memory (LSTM) network. The approach extracts a set of low-level audio and visual features from video data and projects them synchronously into pairs of arousal and valence values using the LSTM network which automatically incorporates the correlations between arousal and valance dimensions. We evaluate the performance of the proposed approach on a dataset comprising video clips segmented from real-world resources such as film, drama, and news, and demonstrate its superior performance over the traditional SVM based method. The results provide one of the earliest preliminary evidence to the benefit of considering correlations between affective dimensions towards accurate AVCA.

READ FULL TEXT

page 4

page 5

research
09/16/2019

Multimodal Deep Models for Predicting Affective Responses Evoked by Movies

The goal of this study is to develop and analyze multimodal models for p...
research
07/21/2017

Shallow reading with Deep Learning: Predicting popularity of online content using only its title

With the ever decreasing attention span of contemporary Internet users, ...
research
06/07/2020

NITS-VC System for VATEX Video Captioning Challenge 2020

Video captioning is process of summarising the content, event and action...
research
12/01/2022

Inference of Media Bias and Content Quality Using Natural-Language Processing

Media bias can significantly impact the formation and development of opi...
research
04/05/2020

Emotional Video to Audio Transformation Using Deep Recurrent Neural Networks and a Neuro-Fuzzy System

Generating music with emotion similar to that of an input video is a ver...
research
09/23/2019

Field typing for improved recognition on heterogeneous handwritten forms

Offline handwriting recognition has undergone continuous progress over t...

Please sign up or login with your details

Forgot password? Click here to reset