Matching-based Term Semantics Pre-training for Spoken Patient Query Understanding

03/02/2023
by   Zefa Hu, et al.
0

Medical Slot Filling (MSF) task aims to convert medical queries into structured information, playing an essential role in diagnosis dialogue systems. However, the lack of sufficient term semantics learning makes existing approaches hard to capture semantically identical but colloquial expressions of terms in medical conversations. In this work, we formalize MSF into a matching problem and propose a Term Semantics Pre-trained Matching Network (TSPMN) that takes both terms and queries as input to model their semantic interaction. To learn term semantics better, we further design two self-supervised objectives, including Contrastive Term Discrimination (CTD) and Matching-based Mask Term Modeling (MMTM). CTD determines whether it is the masked term in the dialogue for each given term, while MMTM directly predicts the masked ones. Experimental results on two Chinese benchmarks show that TSPMN outperforms strong baselines, especially in few-shot settings.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/23/2021

Structural Pre-training for Dialogue Comprehension

Pre-trained language models (PrLMs) have demonstrated superior performan...
research
06/04/2021

Self-supervised Dialogue Learning for Spoken Conversational Question Answering

In spoken conversational question answering (SCQA), the answer to the co...
research
12/12/2022

Momentum Contrastive Pre-training for Question Answering

Existing pre-training methods for extractive Question Answering (QA) gen...
research
05/20/2021

CREAD: Combined Resolution of Ellipses and Anaphora in Dialogues

Anaphora and ellipses are two common phenomena in dialogues. Without res...
research
09/22/2021

DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encoder

With the rapid development of artificial intelligence, conversational bo...
research
08/10/2023

Beyond Semantics: Learning a Behavior Augmented Relevance Model with Self-supervised Learning

Relevance modeling aims to locate desirable items for corresponding quer...
research
07/30/2023

A Knowledge-enhanced Two-stage Generative Framework for Medical Dialogue Information Extraction

This paper focuses on term-status pair extraction from medical dialogues...

Please sign up or login with your details

Forgot password? Click here to reset