EmailSum: Abstractive Email Thread Summarization

by   Shiyue Zhang, et al.

Recent years have brought about an interest in the challenging task of summarizing conversation threads (meetings, online discussions, etc.). Such summaries help analysis of the long text to quickly catch up with the decisions made and thus improve our work or communication efficiency. To spur research in thread summarization, we have developed an abstractive Email Thread Summarization (EmailSum) dataset, which contains human-annotated short (<30 words) and long (<100 words) summaries of 2549 email threads (each containing 3 to 10 emails) over a wide variety of topics. We perform a comprehensive empirical study to explore different summarization techniques (including extractive and abstractive methods, single-document and hierarchical models, as well as transfer and semisupervised learning) and conduct human evaluations on both short and long summary generation tasks. Our results reveal the key challenges of current abstractive summarization models in this task, such as understanding the sender's intent and identifying the roles of sender and receiver. Furthermore, we find that widely used automatic evaluation metrics (ROUGE, BERTScore) are weakly correlated with human judgments on this email thread summarization task. Hence, we emphasize the importance of human evaluation and the development of better metrics by the community. Our code and summary data have been made available at:


page 1

page 2

page 3

page 4


How Far are We from Robust Long Abstractive Summarization?

Abstractive summarization has made tremendous progress in recent years. ...

SummIt: Iterative Text Summarization via ChatGPT

Existing text summarization systems have made significant progress in re...

SummerTime: Text Summarization Toolkit for Non-experts

Recent advances in summarization provide models that can generate summar...

Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization

The problems of unfaithful summaries have been widely discussed under th...

Gold Standard Online Debates Summaries and First Experiments Towards Automatic Summarization of Online Debate Data

Usage of online textual media is steadily increasing. Daily, more and mo...

Nutri-bullets: Summarizing Health Studies by Composing Segments

We introduce Nutri-bullets, a multi-document summarization task for heal...

Summarization Programs: Interpretable Abstractive Summarization with Neural Modular Trees

Current abstractive summarization models either suffer from a lack of cl...

Please sign up or login with your details

Forgot password? Click here to reset