CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation

07/28/2022
by   Hao Sun, et al.
7

Multimodal sentiment analysis and depression estimation are two important research topics that aim to predict human mental states using multimodal data. Previous research has focused on developing effective fusion strategies for exchanging and integrating mind-related information from different modalities. Some MLP-based techniques have recently achieved considerable success in a variety of computer vision tasks. Inspired by this, we explore multimodal approaches with a feature-mixing perspective in this study. To this end, we introduce CubeMLP, a multimodal feature processing framework based entirely on MLP. CubeMLP consists of three independent MLP units, each of which has two affine transformations. CubeMLP accepts all relevant modality features as input and mixes them across three axes. After extracting the characteristics using CubeMLP, the mixed multimodal features are flattened for task predictions. Our experiments are conducted on sentiment analysis datasets: CMU-MOSI and CMU-MOSEI, and depression estimation dataset: AVEC2019. The results show that CubeMLP can achieve state-of-the-art performance with a much lower computing cost.

READ FULL TEXT
research
07/23/2017

Tensor Fusion Network for Multimodal Sentiment Analysis

Multimodal sentiment analysis is an increasingly popular research area, ...
research
11/12/2022

Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts

Multimodal sentiment analysis is a trending topic with the explosion of ...
research
06/27/2023

ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis

Multimodal Sentiment Analysis leverages multimodal signals to detect the...
research
07/20/2023

General Debiasing for Multimodal Sentiment Analysis

Existing work on Multimodal Sentiment Analysis (MSA) utilizes multimodal...
research
03/03/2021

Video Sentiment Analysis with Bimodal Information-augmented Multi-Head Attention

Sentiment analysis is the basis of intelligent human-computer interactio...
research
07/14/2018

GPU-based Commonsense Paradigms Reasoning for Real-Time Query Answering and Multimodal Analysis

We utilize commonsense knowledge bases to address the problem of real- t...
research
03/26/2023

Exploring Multimodal Sentiment Analysis via CBAM Attention and Double-layer BiLSTM Architecture

Because multimodal data contains more modal information, multimodal sent...

Please sign up or login with your details

Forgot password? Click here to reset