Detecting Hallucinated Content in Conditional Neural Sequence Generation

11/05/2020
by   Chunting Zhou, et al.
2

Neural sequence models can generate highly fluent sentences but recent studies have also shown that they are also prone to hallucinate additional content not supported by the input, which can cause a lack of trust in the model. To better assess the faithfulness of the machine outputs, we propose a new task to predict whether each token in the output sequence is hallucinated conditioned on the source input, and collect new manually annotated evaluation sets for this task. We also introduce a novel method for learning to model hallucination detection, based on pretrained language models fine tuned on synthetic data that includes automatically inserted hallucinations. Experiments on machine translation and abstract text summarization demonstrate the effectiveness of our proposed approach – we obtain an average F1 of around 0.6 across all the benchmark datasets and achieve significant improvements in sentence-level hallucination scoring compared to baseline methods. We also release our annotated data and code for future research at https://github.com/violet-zct/fairseq-detect-hallucination.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/29/2021

CoDesc: A Large Code-Description Parallel Dataset

Translation between natural language and source code can help software d...
04/07/2020

Salience Estimation with Multi-Attention Learning for Abstractive Text Summarization

Attention mechanism plays a dominant role in the sequence generation mod...
06/03/2021

Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models

This paper studies how to automatically generate a natural language text...
06/21/2022

BenchCLAMP: A Benchmark for Evaluating Language Models on Semantic Parsing

We introduce BenchCLAMP, a Benchmark to evaluate Constrained LAnguage Mo...
07/28/2022

Sequential Models in the Synthetic Data Vault

The goal of this paper is to describe a system for generating synthetic ...
04/26/2022

PLOD: An Abbreviation Detection Dataset for Scientific Documents

The detection and extraction of abbreviations from unstructured texts ca...
04/27/2022

An End-to-End Dialogue Summarization System for Sales Calls

Summarizing sales calls is a routine task performed manually by salespeo...