FaVIQ: FAct Verification from Information-seeking Questions

07/05/2021
by   Jungsoo Park, et al.
0

Despite significant interest in developing general purpose fact checking models, it is challenging to construct a large-scale fact verification dataset with realistic claims that would occur in the real world. Existing claims are either authored by crowdworkers, thereby introducing subtle biases that are difficult to control for, or manually verified by professional fact checkers, causing them to be expensive and limited in scale. In this paper, we construct a challenging, realistic, and large-scale fact verification dataset called FaVIQ, using information-seeking questions posed by real users who do not know how to answer. The ambiguity in information-seeking questions enables automatically constructing true and false claims that reflect confusions arisen from users (e.g., the year of the movie being filmed vs. being released). Our claims are verified to be natural, contain little lexical bias, and require a complete understanding of the evidence for verification. Our experiments show that the state-of-the-art models are far from solving our new task. Moreover, training on our data helps in professional fact-checking, outperforming models trained on the most widely used dataset FEVER or in-domain data by up to 17 absolute. Altogether, our data will serve as a challenging benchmark for natural language understanding and support future progress in professional fact checking.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/25/2022

Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation

Misinformation emerges in times of uncertainty when credible information...
research
03/13/2021

Automated Fact-Checking for Assisting Human Fact-Checkers

The reporting and analysis of current events around the globe has expand...
research
06/17/2021

X-FACT: A New Benchmark Dataset for Multilingual Fact Checking

In this work, we introduce X-FACT: the largest publicly available multil...
research
06/10/2021

FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information

Fact verification has attracted a lot of attention in the machine learni...
research
05/07/2023

FACTIFY-5WQA: 5W Aspect-based Fact Verification through Question Answering

Automatic fact verification has received significant attention recently....
research
03/14/2020

Scrutinizer: A Mixed-Initiative Approach to Large-Scale, Data-Driven Claim Verification

Organizations such as the International Energy Agency (IEA) spend signif...
research
02/15/2023

COVID-VTS: Fact Extraction and Verification on Short Video Platforms

We introduce a new benchmark, COVID-VTS, for fact-checking multi-modal i...

Please sign up or login with your details

Forgot password? Click here to reset