DialFact: A Benchmark for Fact-Checking in Dialogue

10/15/2021
by   Prakhar Gupta, et al.
4

Fact-checking is an essential tool to mitigate the spread of misinformation and disinformation, however, it has been often explored to verify formal single-sentence claims instead of casual conversational claims. To study the problem, we introduce the task of fact-checking in dialogue. We construct DialFact, a testing benchmark dataset of 22,245 annotated conversational claims, paired with pieces of evidence from Wikipedia. There are three sub-tasks in DialFact: 1) Verifiable claim detection task distinguishes whether a response carries verifiable factual information; 2) Evidence retrieval task retrieves the most relevant Wikipedia snippets as evidence; 3) Claim verification task predicts a dialogue response to be supported, refuted, or not enough information. We found that existing fact-checking models trained on non-dialogue data like FEVER fail to perform well on our task, and thus, we propose a simple yet data-efficient solution to effectively improve fact-checking performance in dialogue. We point out unique challenges in DialFact such as handling the colloquialisms, coreferences, and retrieval ambiguities in the error analysis to shed light on future research in this direction.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 10

04/30/2020

Fact or Fiction: Verifying Scientific Claims

We introduce the task of scientific fact-checking. Given a corpus of sci...
04/16/2021

WhatTheWikiFact: Fact-Checking Claims Against Wikipedia

The rise of Internet has made it a major source of information. Unfortun...
04/27/2020

DeSePtion: Dual Sequence Prediction and Adversarial Examples for Improved Fact-Checking

The increased focus on misinformation has spurred development of data an...
12/27/2018

Uni-DUE Student Team: Tackling fact checking through decomposable attention neural network

In this paper we present our system for the FEVER Challenge. The task of...
09/10/2020

Multi-Hop Fact Checking of Political Claims

Recently, novel multi-hop models and datasets have been introduced to ac...
06/07/2019

FAKTA: An Automatic End-to-End Fact Checking System

We present FAKTA which is a unified framework that integrates various co...
06/02/2021

Evidence-based Factual Error Correction

This paper introduces the task of factual error correction: performing e...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.