Team Triple-Check at Factify 2: Parameter-Efficient Large Foundation Models with Feature Representations for Multi-Modal Fact Verification

02/12/2023
by   Wei-Wei Du, et al.
0

Multi-modal fact verification has become an important but challenging issue on social media due to the mismatch between the text and images in the misinformation of news content, which has been addressed by considering cross-modalities to identify the veracity of the news in recent years. In this paper, we propose the Pre-CoFactv2 framework with new parameter-efficient foundation models for modeling fine-grained text and input embeddings with lightening parameters, multi-modal multi-type fusion for not only capturing relations for the same and different modalities but also for different types (i.e., claim and document), and feature representations for explicitly providing metadata for each sample. In addition, we introduce a unified ensemble method to boost model performance by adjusting the importance of each trained model with not only the weights but also the powers. Extensive experiments show that Pre-CoFactv2 outperforms Pre-CoFact by a large margin and achieved new state-of-the-art results at the Factify challenge at AAAI 2023. We further illustrate model variations to verify the relative contributions of different components. Our team won the first prize (F1-score: 81.82 made our code publicly available at https://github.com/wwweiwei/Pre-CoFactv2-AAAI-2023.

READ FULL TEXT

page 2

page 11

research
01/26/2022

Team Yao at Factify 2022: Utilizing Pre-trained Models and Co-attention Networks for Multi-Modal Fact Verification

In recent years, social media has enabled users to get exposed to a myri...
research
08/23/2022

Multi-Modal Representation Learning with Self-Adaptive Thresholds for Commodity Verification

In this paper, we propose a method to identify identical commodities. In...
research
03/20/2023

Visual Prompt Multi-Modal Tracking

Visible-modal object tracking gives rise to a series of downstream multi...
research
12/16/2021

Logically at Factify 2022: Multimodal Fact Verification

This paper describes our participant system for the multi-modal fact ver...
research
04/03/2023

Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusion

The easy sharing of multimedia content on social media has caused a rapi...
research
06/18/2023

Focusing on Relevant Responses for Multi-modal Rumor Detection

In the absence of an authoritative statement about a rumor, people may e...
research
04/12/2023

Open-TransMind: A New Baseline and Benchmark for 1st Foundation Model Challenge of Intelligent Transportation

With the continuous improvement of computing power and deep learning alg...

Please sign up or login with your details

Forgot password? Click here to reset