Generating medically-accurate summaries of patient-provider dialogue: A multi-stage approach using large language models

by   Varun Nair, et al.

A medical provider's summary of a patient visit serves several critical purposes, including clinical decision-making, facilitating hand-offs between providers, and as a reference for the patient. An effective summary is required to be coherent and accurately capture all the medically relevant information in the dialogue, despite the complexity of patient-generated language. Even minor inaccuracies in visit summaries (for example, summarizing "patient does not have a fever" when a fever is present) can be detrimental to the outcome of care for the patient. This paper tackles the problem of medical conversation summarization by discretizing the task into several smaller dialogue-understanding tasks that are sequentially built upon. First, we identify medical entities and their affirmations within the conversation to serve as building blocks. We study dynamically constructing few-shot prompts for tasks by conditioning on relevant patient information and use GPT-3 as the backbone for our experiments. We also develop GPT-derived summarization metrics to measure performance against reference summaries quantitatively. Both our human evaluation study and metrics for medical correctness show that summaries generated using this approach are clinically accurate and outperform the baseline approach of summarizing the dialog in a zero-shot, single-prompt setting.


ED-FAITH: Evaluating Dialogue Summarization on Faithfulness

Abstractive summarization models typically generate content unfaithful t...

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts

Sifting through vast textual data and summarizing key information impose...

Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures

Understanding a medical conversation between a patient and a physician p...

Adding more data does not always help: A study in medical conversation summarization with PEGASUS

Medical conversation summarization is integral in capturing information ...

SummQA at MEDIQA-Chat 2023:In-Context Learning with GPT-4 for Medical Summarization

Medical dialogue summarization is challenging due to the unstructured na...

Summaries, Highlights, and Action items: Design, implementation and evaluation of an LLM-powered meeting recap system

Meetings play a critical infrastructural role in the coordination of wor...

Heuristic-based Inter-training to Improve Few-shot Multi-perspective Dialog Summarization

Many organizations require their customer-care agents to manually summar...

Please sign up or login with your details

Forgot password? Click here to reset