Log In Sign Up

Large-Context Conversational Representation Learning: Self-Supervised Learning for Conversational Documents

by   Ryo Masumura, et al.

This paper presents a novel self-supervised learning method for handling conversational documents consisting of transcribed text of human-to-human conversations. One of the key technologies for understanding conversational documents is utterance-level sequential labeling, where labels are estimated from the documents in an utterance-by-utterance manner. The main issue with utterance-level sequential labeling is the difficulty of collecting labeled conversational documents, as manual annotations are very costly. To deal with this issue, we propose large-context conversational representation learning (LC-CRL), a self-supervised learning method specialized for conversational documents. A self-supervised learning task in LC-CRL involves the estimation of an utterance using all the surrounding utterances based on large-context language modeling. In this way, LC-CRL enables us to effectively utilize unlabeled conversational documents and thereby enhances the utterance-level sequential labeling. The results of experiments on scene segmentation tasks using contact center conversational datasets demonstrate the effectiveness of the proposed method.


page 1

page 2

page 3

page 4


Conversational Query Rewriting with Self-supervised Learning

Context modeling plays a critical role in building multi-turn dialogue s...

Improving Limited Labeled Dialogue State Tracking with Self-Supervision

Existing dialogue state tracking (DST) models require plenty of labeled ...

Automated Utterance Labeling of Conversations Using Natural Language Processing

Conversational data is essential in psychology because it can help resea...

Self-supervised Dialogue Learning for Spoken Conversational Question Answering

In spoken conversational question answering (SCQA), the answer to the co...

Attentional Multi-Reading Sarcasm Detection

Recognizing sarcasm often requires a deep understanding of multiple sour...

Self-supervised Deep Reconstruction of Mixed Strip-shredded Text Documents

The reconstruction of shredded documents consists of coherently arrangin...

Talking to myself: self-dialogues as data for conversational agents

Conversational agents are gaining popularity with the increasing ubiquit...