DialFact: A Benchmark for Fact-Checking in Dialogue

by   Prakhar Gupta, et al.

Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation, however, it has been often explored to verify formal single-sentence claims instead of casual conversational claims. To study the problem, we introduce the task of fact-checking in dialogue. We construct DialFact, a testing benchmark dataset of 22,245 annotated conversational claims, paired with pieces of evidence from Wikipedia. There are three sub-tasks in DialFact: 1) Verifiable claim detection task distinguishes whether a response carries verifiable factual information; 2) Evidence retrieval task retrieves the most relevant Wikipedia snippets as evidence; 3) Claim verification task predicts a dialogue response to be supported, refuted, or not enough information. We found that existing fact-checking models trained on non-dialogue data like FEVER fail to perform well on our task, and thus, we propose a simple yet data-efficient solution to effectively improve fact-checking performance in dialogue. We point out unique challenges in DialFact such as handling the colloquialisms, coreferences, and retrieval ambiguities in the error analysis to shed light on future research in this direction.



There are no comments yet.


page 10


Fact or Fiction: Verifying Scientific Claims

We introduce the task of scientific fact-checking. Given a corpus of sci...

WhatTheWikiFact: Fact-Checking Claims Against Wikipedia

The rise of Internet has made it a major source of information. Unfortun...

DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking

The increased focus on misinformation has spurred development of data an...

Uni-DUE Student Team: Tackling fact checking through decomposable attention neural network

In this paper we present our system for the FEVER Challenge. The task of...

Multi-Hop Fact Checking of Political Claims

Recently, novel multi-hop models and datasets have been introduced to ac...

FAKTA: An Automatic End-to-End Fact Checking System

We present FAKTA which is a unified framework that integrates various co...

Evidence-based Factual Error Correction

This paper introduces the task of factual error correction: performing e...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.