Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework

06/14/2022
by   Ahmed Mahran, et al.
0

In this paper we describe our work towards building a generic framework for both multi-modal embedding and multi-label binary classification tasks, while participating in task 5 (Multimedia Automatic Misogyny Identification) of SemEval 2022 competition. Since pretraining deep models from scratch is a resource and data hungry task, our approach is based on three main strategies. We combine different state-of-the-art architectures to capture a wide spectrum of semantic signals from the multi-modal input. We employ a multi-task learning scheme to be able to use multiple datasets from the same knowledge domain to help increase the model's performance. We also use multiple objectives to regularize and fine tune different system components.

READ FULL TEXT
research
06/16/2023

Multi-task 3D building understanding with multi-modal pretraining

This paper explores various learning strategies for 3D building type cla...
research
01/22/2022

A Multi-modal Fusion Framework Based on Multi-task Correlation Learning for Cancer Prognosis Prediction

Morphological attributes from histopathological images and molecular pro...
research
12/08/2022

OFASys: A Multi-Modal Multi-Task Learning System for Building Generalist Models

Generalist models, which are capable of performing diverse multi-modal t...
research
04/24/2020

Bharatanatyam Dance Transcription using Multimedia Ontology and Machine Learning

Indian Classical Dance is an over 5000 years' old multi-modal language f...
research
09/16/2021

Overview of Tencent Multi-modal Ads Video Understanding Challenge

Multi-modal Ads Video Understanding Challenge is the first grand challen...
research
04/17/2021

Semi-Supervised Multi-Modal Multi-Instance Multi-Label Deep Network with Optimal Transport

Complex objects are usually with multiple labels, and can be represented...

Please sign up or login with your details

Forgot password? Click here to reset