DeepAI AI Chat
Log In Sign Up

Multimodal Emotion-Cause Pair Extraction in Conversations

by   Fanfan Wang, et al.
Nanjing University

Emotion cause analysis has received considerable attention in recent years. Previous studies primarily focused on emotion cause extraction from texts in news articles or microblogs. It is also interesting to discover emotions and their causes in conversations. As conversation in its natural form is multimodal, a large number of studies have been carried out on multimodal emotion recognition in conversations, but there is still a lack of work on multimodal emotion cause analysis. In this work, we introduce a new task named Multimodal Emotion-Cause Pair Extraction in Conversations, aiming to jointly extract emotions and their associated causes from conversations reflected in multiple modalities (text, audio and video). We accordingly construct a multimodal conversational emotion cause dataset, Emotion-Cause-in-Friends, which contains 9,272 multimodal emotion-cause pairs annotated on 13,509 utterances in the sitcom Friends. We finally benchmark the task by establishing a baseline system that incorporates multimodal features for emotion-cause pair extraction. Preliminary experimental results demonstrate the potential of multimodal information fusion for discovering both emotions and causes in conversations.


page 2

page 4


Emotion-Cause Pair Extraction: A New Task to Emotion Analysis in Texts

Emotion cause extraction (ECE), the task aimed at extracting the potenti...

MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations

Emotion recognition in conversations is a challenging Artificial Intelli...

Recognizing Emotion Cause in Conversations

Recognizing the cause behind emotions in text is a fundamental yet under...

Multi-Task Learning Framework for Extracting Emotion Cause Span and Entailment in Conversations

Predicting emotions expressed in text is a well-studied problem in the N...

A Dual-Questioning Attention Network for Emotion-Cause Pair Extraction with Context Awareness

Emotion-cause pair extraction (ECPE), an emerging task in sentiment anal...

Detecting Concept-level Emotion Cause in Microblogging

In this paper, we propose a Concept-level Emotion Cause Model (CECM), in...

Empathic Conversations: A Multi-level Dataset of Contextualized Conversations

Empathy is a cognitive and emotional reaction to an observed situation o...