Adding more data does not always help: A study in medical conversation summarization with PEGASUS

by   Varun Nair, et al.

Medical conversation summarization is integral in capturing information gathered during interactions between patients and physicians. Summarized conversations are used to facilitate patient hand-offs between physicians, and as part of providing care in the future. Summaries, however, can be time-consuming to produce and require domain expertise. Modern pre-trained NLP models such as PEGASUS have emerged as capable alternatives to human summarization, reaching state-of-the-art performance on many summarization benchmarks. However, many downstream tasks still require at least moderately sized datasets to achieve satisfactory performance. In this work we (1) explore the effect of dataset size on transfer learning medical conversation summarization using PEGASUS and (2) evaluate various iterative labeling strategies in the low-data regime, following their success in the classification setting. We find that model performance saturates with increase in dataset size and that the various active-learning strategies evaluated all show equivalent performance consistent with simple dataset size increase. We also find that naive iterative pseudo-labeling is on-par or slightly worse than no pseudo-labeling. Our work sheds light on the successes and challenges of translating low-data regime techniques in classification to medical conversation summarization and helps guides future work in this space. Relevant code available at <>.


Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures

Understanding a medical conversation between a patient and a physician p...

Learning functional sections in medical conversations: iterative pseudo-labeling and human-in-the-loop approach

Medical conversations between patients and medical professionals have im...

Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models

A medical provider's summary of a patient visit serves several critical ...

Unsupervised Learning of Hierarchical Conversation Structure

Human conversations can evolve in many different ways, creating challeng...

Structure-Aware Abstractive Conversation Summarization via Discourse and Action Graphs

Abstractive conversation summarization has received much attention recen...

IUTEAM1 at MEDIQA-Chat 2023: Is simple fine tuning effective for multilayer summarization of clinical conversations?

Clinical conversation summarization has become an important application ...

Towards Ontological Conversation Interpretation: A Method for Ontology Creation from Medical Guidelines

The automated capturing and summarization of medical consultations aims ...

Please sign up or login with your details

Forgot password? Click here to reset