Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses

09/08/2019
by   Matt Grenander, et al.
0

Sentence position is a strong feature for news summarization, since the lead often (but not always) summarizes the key points of the article. In this paper, we show that recent neural systems excessively exploit this trend, which although powerful for many inputs, is also detrimental when summarizing documents where important content should be extracted from later parts of the article. We propose two techniques to make systems sensitive to the importance of content in different parts of the article. The first technique employs 'unbiased' data; i.e., randomly shuffled sentences of the source document, to pretrain the model. The second technique uses an auxiliary ROUGE-based loss that encourages the model to distribute importance scores throughout a document by mimicking sentence-level ROUGE scores on the training data. We show that these techniques significantly improve the performance of a competitive reinforcement learning based extractive system, with the auxiliary loss being more powerful than pretraining.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/26/2017

Detecting (Un)Important Content for Single-Document News Summarization

We present a robust approach for detecting intrinsic sentence importance...
research
03/19/2022

Read Top News First: A Document Reordering Approach for Multi-Document News Summarization

A common method for extractive multi-document news summarization is to r...
research
04/22/2018

Neural Sentence Location Prediction for Summarization

A competitive baseline in sentence-level extractive summarization of new...
research
12/25/2019

Make Lead Bias in Your Favor: A Simple and Effective Method for News Summarization

Lead bias is a common phenomenon in news summarization, where early part...
research
05/29/2021

Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning

In news articles the lead bias is a common phenomenon that usually domin...
research
12/31/2022

Towards Proactively Forecasting Sentence-Specific Information Popularity within Online News Documents

Multiple studies have focused on predicting the prospective popularity o...
research
07/13/2022

A General Contextualized Rewriting Framework for Text Summarization

The rewriting method for text summarization combines extractive and abst...

Please sign up or login with your details

Forgot password? Click here to reset