Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthiness

08/08/2018
by   Pepa Atanasova, et al.
0

We present an overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims, with focus on Task 1: Check-Worthiness. The task asks to predict which claims in a political debate should be prioritized for fact-checking. In particular, given a debate or a political speech, the goal was to produce a ranked list of its sentences based on their worthiness for fact checking. We offered the task in both English and Arabic, based on debates from the 2016 US Presidential Campaign, as well as on some speeches during and after the campaign. A total of 30 teams registered to participate in the Lab and seven teams actually submitted systems for Task 1. The most successful approaches used by the participants relied on recurrent and multi-layer neural networks, as well as on combinations of distributional representations, on matchings claims' vocabulary against lexicons, and on measures of syntactic dependency. The best systems achieved mean average precision of 0.18 and 0.15 on the English and on the Arabic test datasets, respectively. This leaves large room for further improvement, and thus we release all datasets and the scoring scripts, which should enable further research in check-worthiness estimation.

READ FULL TEXT
research
07/15/2020

Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media

We present an overview of the third edition of the CheckThat! Lab at CLE...
research
04/17/2020

Too Many Claims to Fact-Check: Prioritizing Political Claims Based on Check-Worthiness

The massive amount of misinformation spreading on the Internet on a dail...
research
01/21/2020

CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media

We describe the third edition of the CheckThat! Lab, which is part of th...
research
08/04/2019

Automatic Fact-Checking Using Context and Discourse Information

We study the problem of automatic fact-checking, paying special attentio...
research
08/01/2020

Overview of CLEF 2019 Lab ProtestNews: Extracting Protests from News in a Cross-context Setting

We present an overview of the CLEF-2019 Lab ProtestNews on Extracting Pr...
research
09/19/2021

UPV at CheckThat! 2021: Mitigating Cultural Differences for Identifying Multilingual Check-worthy Claims

Identifying check-worthy claims is often the first step of automated fac...

Please sign up or login with your details

Forgot password? Click here to reset