Is this Dialogue Coherent? Learning from Dialogue Acts and Entities

06/17/2020
by   Alessandra Cervone, et al.
0

In this work, we investigate the human perception of coherence in open-domain dialogues. In particular, we address the problem of annotating and modeling the coherence of next-turn candidates while considering the entire history of the dialogue. First, we create the Switchboard Coherence (SWBD-Coh) corpus, a dataset of human-human spoken dialogues annotated with turn coherence ratings, where next-turn candidate utterances ratings are provided considering the full dialogue context. Our statistical analysis of the corpus indicates how turn coherence perception is affected by patterns of distribution of entities previously introduced and the Dialogue Acts used. Second, we experiment with different architectures to model entities, Dialogue Acts and their combination and evaluate their performance in predicting human coherence ratings on SWBD-Coh. We find that models combining both DA and entity information yield the best performances both for response selection and turn coherence rating.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/21/2021

Modeling Performance in Open-Domain Dialogue with PARADISE

There has recently been an explosion of work on spoken dialogue systems,...
research
05/29/2023

Medical Dialogue Generation via Dual Flow Modeling

Medical dialogue systems (MDS) aim to provide patients with medical serv...
research
06/21/2018

Coherence Models for Dialogue

Coherence across multiple turns is a major challenge for state-of-the-ar...
research
05/18/2020

Neural Generation of Dialogue Response Timings

The timings of spoken response offsets in human dialogue have been shown...
research
09/23/2019

Towards Best Experiment Design for Evaluating Dialogue System Output

To overcome the limitations of automated metrics (e.g. BLEU, METEOR) for...
research
06/01/2021

Towards Quantifiable Dialogue Coherence Evaluation

Automatic dialogue coherence evaluation has attracted increasing attenti...
research
08/22/2019

A Neural Model for Dialogue Coherence Assessment

Dialogue quality assessment is crucial for evaluating dialogue agents. A...

Please sign up or login with your details

Forgot password? Click here to reset