Uncertainty-Aware Abstractive Summarization

05/21/2021
by   Alexios Gidiotis, et al.
0

We propose a novel approach to summarization based on Bayesian deep learning. We approximate Bayesian summary generation by first extending state-of-the-art summarization models with Monte Carlo dropout and then using them to perform multiple stochastic forward passes. This method allows us to improve summarization performance by simply using the median of multiple stochastic summaries. We show that our variational equivalents of BART and PEGASUS can outperform their deterministic counterparts on multiple benchmark datasets. In addition, we rely on Bayesian inference to measure the uncertainty of the model when generating summaries. Having a reliable uncertainty measure, we can improve the experience of the end user by filtering out generated summaries of high uncertainty. Furthermore, our proposed metric could be used as a criterion for selecting samples for annotation, and can be paired nicely with active learning and human-in-the-loop approaches.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/17/2023

On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Study

Modern deep models for summarization attains impressive benchmark perfor...
research
05/16/2019

Loop Summarization with Rational Vector Addition Systems (extended version)

This paper presents a technique for computing numerical loop summaries. ...
research
10/09/2021

Bayesian Active Summarization

Bayesian Active Learning has had significant impact to various NLP probl...
research
05/10/2021

Improving Factual Consistency of Abstractive Summarization via Question Answering

A commonly observed problem with the state-of-the art abstractive summar...
research
04/27/2021

Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics

Modern summarization models generate highly fluent but often factually u...
research
03/01/2019

Video Summarization via Actionness Ranking

To automatically produce a brief yet expressive summary of a long video,...
research
04/18/2019

Point-less: More Abstractive Summarization with Pointer-Generator Networks

The Pointer-Generator architecture has shown to be a big improvement for...

Please sign up or login with your details

Forgot password? Click here to reset