Multimodal Emotion-Cause Pair Extraction in Conversations

10/15/2021
by   Fanfan Wang, et al.
0

Emotion cause analysis has received considerable attention in recent years. Previous studies primarily focused on emotion cause extraction from texts in news articles or microblogs. It is also interesting to discover emotions and their causes in conversations. As conversation in its natural form is multimodal, a large number of studies have been carried out on multimodal emotion recognition in conversations, but there is still a lack of work on multimodal emotion cause analysis. In this work, we introduce a new task named Multimodal Emotion-Cause Pair Extraction in Conversations, aiming to jointly extract emotions and their associated causes from conversations reflected in multiple modalities (text, audio and video). We accordingly construct a multimodal conversational emotion cause dataset, Emotion-Cause-in-Friends, which contains 9,272 multimodal emotion-cause pairs annotated on 13,509 utterances in the sitcom Friends. We finally benchmark the task by establishing a baseline system that incorporates multimodal features for emotion-cause pair extraction. Preliminary experimental results demonstrate the potential of multimodal information fusion for discovering both emotions and causes in conversations.

READ FULL TEXT

page 2

page 4

research
06/04/2019

Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts

Emotion cause extraction (ECE), the task aimed at extracting the potenti...
research
10/05/2018

MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations

Emotion recognition in conversations is a challenging Artificial Intelli...
research
08/31/2023

Socratis: Are large multimodal models emotionally aware?

Existing emotion prediction benchmarks contain coarse emotion labels whi...
research
12/22/2020

Recognizing Emotion Cause in Conversations

Recognizing the cause behind emotions in text is a fundamental yet under...
research
11/07/2022

Multi-Task Learning Framework for Extracting Emotion Cause Span and Entailment in Conversations

Predicting emotions expressed in text is a well-studied problem in the N...
research
04/15/2021

A Dual-Questioning Attention Network for Emotion-Cause Pair Extraction with Context Awareness

Emotion-cause pair extraction (ECPE), an emerging task in sentiment anal...
research
04/30/2015

Detecting Concept-level Emotion Cause in Microblogging

In this paper, we propose a Concept-level Emotion Cause Model (CECM), in...

Please sign up or login with your details

Forgot password? Click here to reset