CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization

09/19/2021
by   Shuyang Cao, et al.
0

We study generating abstractive summaries that are faithful and factually consistent with the given articles. A novel contrastive learning formulation is presented, which leverages both reference summaries, as positive training data, and automatically generated erroneous summaries, as negative training data, to train summarization systems that are better at distinguishing between them. We further design four types of strategies for creating negative samples, to resemble errors made commonly by two state-of-the-art models, BART and PEGASUS, found in our new human annotations of summary errors. Experiments on XSum and CNN/Daily Mail show that our contrastive learning framework is robust across datasets and models. It consistently produces more factual summaries than strong comparisons with post error correction, entailment-based reranking, and unlikelihood training, according to QA-based factuality evaluation. Human judges echo the observation and find that our model summaries correct more errors.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/05/2020

Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning

Evaluation of a document summarization system has been a critical factor...
research
07/10/2023

Improving Factuality of Abstractive Summarization via Contrastive Reward Learning

Modern abstractive summarization models often generate summaries that co...
research
10/22/2022

Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling

Abstractive summarization models often generate inconsistent summaries c...
research
09/29/2022

COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization

Traditional training paradigms for extractive and abstractive summarizat...
research
12/02/2021

CO2Sum:Contrastive Learning for Factual-Consistent Abstractive Summarization

Generating factual-consistent summaries is a challenging task for abstra...
research
02/08/2020

Storyboard: Optimizing Precomputed Summaries for Aggregation

An emerging class of data systems partition their data and precompute ap...
research
10/17/2020

Factual Error Correction for Abstractive Summarization Models

Neural abstractive summarization systems have achieved promising progres...

Please sign up or login with your details

Forgot password? Click here to reset