Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization

04/28/2022
by   Yue Dong, et al.
0

Despite recent advances in abstractive summarization, current summarization systems still suffer from content hallucinations where models generate text that is either irrelevant or contradictory to the source document. However, prior work has been predicated on the assumption that any generated facts not appearing explicitly in the source are undesired hallucinations. Methods have been proposed to address this scenario by ultimately improving `faithfulness' to the source document, but in reality, there is a large portion of entities in the gold reference targets that are not directly in the source. In this work, we show that these entities are not aberrations, but they instead require utilizing external world knowledge to infer reasoning paths from entities in the source. We show that by utilizing an external knowledge base, we can improve the faithfulness of summaries without simply making them more extractive, and additionally, we show that external knowledge bases linked from the source can benefit the factuality of generated summaries.

READ FULL TEXT
research
05/14/2023

FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge

Evaluating the factual consistency of automatically generated summaries ...
research
06/14/2018

Entity Commonsense Representation for Neural Abstractive Summarization

A major proportion of a text summary includes important entities found i...
research
09/07/2022

Entity-based SpanCopy for Abstractive Summarization to Improve the Factual Consistency

Despite the success of recent abstractive summarizers on automatic evalu...
research
03/30/2022

Entity-driven Fact-aware Abstractive Summarization of Biomedical Literature

As part of the large number of scientific articles being published every...
research
06/04/2019

HighRES: Highlight-based Reference-less Evaluation of Summarization

There has been substantial progress in summarization research enabled by...
research
08/23/2018

Improving Abstraction in Text Summarization

Abstractive text summarization aims to shorten long text documents into ...
research
08/31/2021

Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization

Despite recent progress in abstractive summarization, systems still suff...

Please sign up or login with your details

Forgot password? Click here to reset