Uncertainty-Aware Abstractive Summarization

05/21/2021
by   Alexios Gidiotis, et al.
0

We propose a novel approach to summarization based on Bayesian deep learning. We approximate Bayesian summary generation by first extending state-of-the-art summarization models with Monte Carlo dropout and then using them to perform multiple stochastic forward passes. This method allows us to improve summarization performance by simply using the median of multiple stochastic summaries. We show that our variational equivalents of BART and PEGASUS can outperform their deterministic counterparts on multiple benchmark datasets. In addition, we rely on Bayesian inference to measure the uncertainty of the model when generating summaries. Having a reliable uncertainty measure, we can improve the experience of the end user by filtering out generated summaries of high uncertainty. Furthermore, our proposed metric could be used as a criterion for selecting samples for annotation, and can be paired nicely with active learning and human-in-the-loop approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/09/2021

Bayesian Active Summarization

Bayesian Active Learning has had significant impact to various NLP probl...
05/16/2019

Loop Summarization with Rational Vector Addition Systems (extended version)

This paper presents a technique for computing numerical loop summaries. ...
05/10/2021

Improving Factual Consistency of Abstractive Summarization via Question Answering

A commonly observed problem with the state-of-the art abstractive summar...
03/19/2020

Boosting Factual Correctness of Abstractive Summarization

A commonly observed problem with abstractive summarization is the distor...
04/27/2021

Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics

Modern summarization models generate highly fluent but often factually u...
05/17/2019

Model interpretation through lower-dimensional posterior summarization

Nonparametric regression models have recently surged in their power and ...
03/01/2019

Video Summarization via Actionness Ranking

To automatically produce a brief yet expressive summary of a long video,...