Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system

by   Sumit Asthana, et al.

Meetings play a critical infrastructural role in the coordination of work. In recent years, due to shift to hybrid and remote work, more meetings are moving to online Computer Mediated Spaces. This has led to new problems (e.g. more time spent in less engaging meetings) and new opportunities (e.g. automated transcription/captioning and recap support). Recent advances in large language models (LLMs) for dialog summarization have the potential to improve the experience of meetings by reducing individuals' meeting load and increasing the clarity and alignment of meeting outputs. Despite this potential, they face technological limitation due to long transcripts and inability to capture diverse recap needs based on user's context. To address these gaps, we design, implement and evaluate in-context a meeting recap system. We first conceptualize two salient recap representations – important highlights, and a structured, hierarchical minutes view. We develop a system to operationalize the representations with dialogue summarization as its building blocks. Finally, we evaluate the effectiveness of the system with seven users in the context of their work meetings. Our findings show promise in using LLM-based dialogue summarization for meeting recap and the need for both representations in different contexts. However, we find that LLM-based recap still lacks an understanding of whats personally relevant to participants, can miss important details, and mis-attributions can be detrimental to group dynamics. We identify collaboration opportunities such as a shared recap document that a high quality recap enables. We report on implications for designing AI systems to partner with users to learn and improve from natural interactions to overcome the limitations related to personal relevance and summarization quality.


page 1

page 2

page 3

page 4


STRUDEL: Structured Dialogue Summarization for Dialogue Comprehension

Abstractive dialogue summarization has long been viewed as an important ...

Evaluating Emotional Nuances in Dialogue Summarization

Automatic dialogue summarization is a well-established task that aims to...

Towards Understanding Omission in Dialogue Summarization

Dialogue summarization aims to condense the lengthy dialogue into a conc...

Benchmarking Large Language Models for News Summarization

Large language models (LLMs) have shown promise for automatic summarizat...

Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models

A medical provider's summary of a patient visit serves several critical ...

Evaluating Older Users' Experiences with Commercial Dialogue Systems: Implications for Future Design and Development

Understanding the needs of a variety of distinct user groups is vital in...

A System for Interleaving Discussion and Summarization in Online Collaboration

In many instances of online collaboration, ideation and deliberation abo...

Please sign up or login with your details

Forgot password? Click here to reset