R2D2: Robust Data-to-Text with Replacement Detection

05/25/2022
by   Linyong Nan, et al.
0

Unfaithful text generation is a common problem for text generation systems. In the case of Data-to-Text (D2T) systems, the factuality of the generated text is particularly crucial for any real-world applications. We introduce R2D2, a training framework that addresses unfaithful Data-to-Text generation by training a system both as a generator and a faithfulness discriminator with additional replacement detection and unlikelihood learning tasks. To facilitate such training, we propose two methods for sampling unfaithful sentences. We argue that the poor entity retrieval capability of D2T systems is one of the primary sources of unfaithfulness, so in addition to the existing metrics, we further propose NER-based metrics to evaluate the fidelity of D2T generations. Our experimental results show that R2D2 systems could effectively mitigate the unfaithful text generation, and they achieve new state-of-the-art results on FeTaQA, LogicNLG, and ToTTo, all with significant improvements.

READ FULL TEXT
research
07/25/2023

Watermarking Conditional Text Generation for AI Detection: Unveiling Challenges and a Semantic-Aware Watermark Remedy

To mitigate potential risks associated with language models, recent AI d...
research
02/01/2017

AMR-to-text Generation with Synchronous Node Replacement Grammar

This paper addresses the task of AMR-to-text generation by leveraging sy...
research
10/19/2019

Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generation

Neural conditional text generation systems have achieved significant pro...
research
07/17/2021

Generative Pretraining for Paraphrase Evaluation

We introduce ParaBLEU, a paraphrase representation learning model and ev...
research
12/18/2022

Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data

As more and more conversational and translation systems are deployed in ...
research
10/24/2022

On the Effectiveness of Automated Metrics for Text Generation Systems

A major challenge in the field of Text Generation is evaluation because ...
research
01/17/2020

Generación automática de frases literarias en español

In this work we present a state of the art in the area of Computational ...

Please sign up or login with your details

Forgot password? Click here to reset