A Multibias-mitigated and Sentiment Knowledge Enriched Transformer for Debiasing in Multimodal Conversational Emotion Recognition

07/17/2022
by   Jinglin Wang, et al.
0

Multimodal emotion recognition in conversations (mERC) is an active research topic in natural language processing (NLP), which aims to predict human's emotional states in communications of multiple modalities, e,g., natural language and facial gestures. Innumerable implicit prejudices and preconceptions fill human language and conversations, leading to the question of whether the current data-driven mERC approaches produce a biased error. For example, such approaches may offer higher emotional scores on the utterances by females than males. In addition, the existing debias models mainly focus on gender or race, where multibias mitigation is still an unexplored task in mERC. In this work, we take the first step to solve these issues by proposing a series of approaches to mitigate five typical kinds of bias in textual utterances (i.e., gender, age, race, religion and LGBTQ+) and visual representations (i.e, gender and age), followed by a Multibias-Mitigated and sentiment Knowledge Enriched bi-modal Transformer (MMKET). Comprehensive experimental results show the effectiveness of the proposed model and prove that the debias operation has a great impact on the classification performance for mERC. We hope our study will benefit the development of bias mitigation in mERC and related emotion studies.

READ FULL TEXT
research
10/05/2018

MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations

Emotion recognition in conversations is a challenging Artificial Intelli...
research
04/09/2021

AdCOFE: Advanced Contextual Feature Extraction in Conversations for emotion classification

Emotion recognition in conversations is an important step in various vir...
research
09/13/2021

Beyond Isolated Utterances: Conversational Emotion Recognition

Speech emotion recognition is the task of recognizing the speaker's emot...
research
05/11/2022

Bias and Fairness on Multimodal Emotion Detection Algorithms

Numerous studies have shown that machine learning algorithms can latch o...
research
01/23/2018

Survey on Emotional Body Gesture Recognition

Automatic emotion recognition has become a trending research topic in th...
research
06/03/2021

DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations

Emotion Recognition in Conversations (ERC) has gained increasing attenti...
research
02/21/2023

ChatGPT: Jack of all trades, master of none

OpenAI has released the Chat Generative Pre-trained Transformer (ChatGPT...

Please sign up or login with your details

Forgot password? Click here to reset