Evidence-based Verification for Real World Information Needs

04/01/2021
by   James Thorne, et al.
0

Claim verification is the task of predicting the veracity of written statements against evidence. Previous large-scale datasets model the task as classification, ignoring the need to retrieve evidence, or are constructed for research purposes, and may not be representative of real-world needs. In this paper, we introduce a novel claim verification dataset with instances derived from search-engine queries, yielding 10,987 claims annotated with evidence that represent real-world information needs. For each claim, we annotate evidence from full Wikipedia articles with both section and sentence-level granularity. Our annotation allows comparison between two complementary approaches to verification: stance classification, and evidence extraction followed by entailment recognition. In our comprehensive evaluation, we find no significant difference in accuracy between these two approaches. This enables systems to use evidence extraction to summarize a rationale for an end-user while maintaining the accuracy when predicting a claim's veracity. With challenging claims and evidence documents containing hundreds of sentences, our dataset presents interesting challenges that are not captured in previous work – evidenced through transfer learning experiments. We release code and data to support further research on this task.

READ FULL TEXT
research
03/14/2018

FEVER: a large-scale dataset for Fact Extraction and VERification

Unlike other tasks and despite recent interest, research in textual clai...
research
04/10/2021

Fool Me Twice: Entailment from Wikipedia Gamification

We release FoolMeTwice (FM2 for short), a large dataset of challenging e...
research
05/02/2023

Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidence

Real-world fact verification task aims to verify the factuality of a cla...
research
09/03/2018

DeFactoNLP: Fact Verification using Entity Recognition, TFIDF Vector Comparison and Decomposable Attention

In this paper, we describe DeFactoNLP, the system we designed for the FE...
research
05/20/2021

Unified Dual-view Cognitive Model for Interpretable Claim Verification

Recent studies constructing direct interactions between the claim and ea...
research
05/19/2023

Complex Claim Verification with Evidence Retrieved in the Wild

Evidence retrieval is a core part of automatic fact-checking. Prior work...
research
04/28/2020

DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification

Recently, many methods discover effective evidence from reliable sources...

Please sign up or login with your details

Forgot password? Click here to reset