TabFact: A Large-scale Dataset for Table-based Fact Verification

09/05/2019
by   Wenhu Chen, et al.
0

The problem of verifying whether a textual hypothesis holds the truth based on the given evidence, also known as fact verification, plays an important role in the study of natural language understanding and semantic representation. However, existing studies are mainly restricted to dealing with unstructured evidence (e.g., natural language sentences and documents, news, etc), while verification under structured evidence, such as tables, graphs, and databases, remains unexplored. This paper specifically aims to study the fact verification given semi-structured data as evidence. To this end, we construct a large-scale dataset called TabFact with 16k Wikipedia tables as evidence for 118k human-annotated natural language statements, which are labeled as either ENTAILED or REFUTED. TabFact is more challenging since it involves both soft linguistic reasoning and hard symbolic reasoning. To address these reasoning challenges, we design two different models: Table-BERT and Latent Program Algorithm (LPA). Table-BERT leverages the state-of-the-art pre-trained language model to encode the linearized tables and statements into continuous vectors for verification. LPA parses statements into LISP-like programs and executes them against the tables to obtain the returned binary value. Both methods achieve similar accuracy but yet far from human performance. We also perform comprehensive analysis and demonstrate great future opportunities. The data and code of the dataset are provided in <https://github.com/wenhuchen/Table-Fact-Checking>.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/21/2021

Sattiy at SemEval-2021 Task 9: An Ensemble Solution for Statement Verification and Evidence Finding with Tables

Question answering from semi-structured tables can be seen as a semantic...
research
09/22/2021

Exploring Decomposition for Table-based Fact Verification

Fact verification based on structured data is challenging as it requires...
research
12/20/2022

Toward a Unified Framework for Unsupervised Complex Tabular Reasoning

Structured tabular data exist across nearly all fields. Reasoning task o...
research
09/09/2021

Table-based Fact Verification with Salience-aware Learning

Tables provide valuable knowledge that can be used to verify textual sta...
research
10/06/2020

Program Enhanced Fact Verification with Verbalization and Graph Attention Network

Performing fact verification based on structured data is important for m...
research
04/19/2022

Table-based Fact Verification with Self-adaptive Mixture of Experts

The table-based fact verification task has recently gained widespread at...
research
01/31/2023

Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning

Table-based reasoning has shown remarkable progress in combining deep mo...

Please sign up or login with your details

Forgot password? Click here to reset