Uncertainty-Aware Abstractive Summarization

by   Alexios Gidiotis, et al.

We propose a novel approach to summarization based on Bayesian deep learning. We approximate Bayesian summary generation by first extending state-of-the-art summarization models with Monte Carlo dropout and then using them to perform multiple stochastic forward passes. This method allows us to improve summarization performance by simply using the median of multiple stochastic summaries. We show that our variational equivalents of BART and PEGASUS can outperform their deterministic counterparts on multiple benchmark datasets. In addition, we rely on Bayesian inference to measure the uncertainty of the model when generating summaries. Having a reliable uncertainty measure, we can improve the experience of the end user by filtering out generated summaries of high uncertainty. Furthermore, our proposed metric could be used as a criterion for selecting samples for annotation, and can be paired nicely with active learning and human-in-the-loop approaches.


page 1

page 2

page 3

page 4


Bayesian Active Summarization

Bayesian Active Learning has had significant impact to various NLP probl...

Loop Summarization with Rational Vector Addition Systems (extended version)

This paper presents a technique for computing numerical loop summaries. ...

Improving Factual Consistency of Abstractive Summarization via Question Answering

A commonly observed problem with the state-of-the art abstractive summar...

Boosting Factual Correctness of Abstractive Summarization

A commonly observed problem with abstractive summarization is the distor...

Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics

Modern summarization models generate highly fluent but often factually u...

Model interpretation through lower-dimensional posterior summarization

Nonparametric regression models have recently surged in their power and ...

Video Summarization via Actionness Ranking

To automatically produce a brief yet expressive summary of a long video,...