DeepAI
Log In Sign Up

Large-Context Conversational Representation Learning: Self-Supervised Learning for Conversational Documents

02/16/2021
by   Ryo Masumura, et al.
0

This paper presents a novel self-supervised learning method for handling conversational documents consisting of transcribed text of human-to-human conversations. One of the key technologies for understanding conversational documents is utterance-level sequential labeling, where labels are estimated from the documents in an utterance-by-utterance manner. The main issue with utterance-level sequential labeling is the difficulty of collecting labeled conversational documents, as manual annotations are very costly. To deal with this issue, we propose large-context conversational representation learning (LC-CRL), a self-supervised learning method specialized for conversational documents. A self-supervised learning task in LC-CRL involves the estimation of an utterance using all the surrounding utterances based on large-context language modeling. In this way, LC-CRL enables us to effectively utilize unlabeled conversational documents and thereby enhances the utterance-level sequential labeling. The results of experiments on scene segmentation tasks using contact center conversational datasets demonstrate the effectiveness of the proposed method.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/09/2021

Conversational Query Rewriting with Self-supervised Learning

Context modeling plays a critical role in building multi-turn dialogue s...
10/26/2020

Improving Limited Labeled Dialogue State Tracking with Self-Supervision

Existing dialogue state tracking (DST) models require plenty of labeled ...
08/12/2022

Automated Utterance Labeling of Conversations Using Natural Language Processing

Conversational data is essential in psychology because it can help resea...
06/04/2021

Self-supervised Dialogue Learning for Spoken Conversational Question Answering

In spoken conversational question answering (SCQA), the answer to the co...
09/09/2018

Attentional Multi-Reading Sarcasm Detection

Recognizing sarcasm often requires a deep understanding of multiple sour...
07/01/2020

Self-supervised Deep Reconstruction of Mixed Strip-shredded Text Documents

The reconstruction of shredded documents consists of coherently arrangin...
09/18/2018

Talking to myself: self-dialogues as data for conversational agents

Conversational agents are gaining popularity with the increasing ubiquit...