I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling

by   Yixin Nie, et al.

To quantify how well natural language understanding models can capture consistency in a general conversation, we introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues. We then compare a structured utterance-based approach of using pre-trained Transformer models for contradiction detection with the typical unstructured approach. Results reveal that: (i) our newly collected dataset is notably more effective at providing supervision for the dialogue contradiction detection task than existing NLI data including those aimed to cover the dialogue domain; (ii) the structured utterance-based approach is more robust and transferable on both analysis and out-of-distribution dialogues than its unstructured counterpart. We also show that our best contradiction detection model correlates well with human judgments and further provide evidence for its usage in both automatically evaluating and improving the consistency of state-of-the-art generative chatbots.


page 8

page 9


Dialogue Natural Language Inference

Consistency is a long standing issue faced by dialogue models. In this p...

Action State Update Approach to Dialogue Management

Utterance interpretation is one of the main functions of a dialogue mana...

Adapting Task-Oriented Dialogue Models for Email Conversations

Intent detection is a key part of any Natural Language Understanding (NL...

Improving Multi-turn Dialogue Modelling with Utterance ReWriter

Recent research has made impressive progress in single-turn dialogue mod...

Learning Locality and Isotropy in Dialogue Modeling

Existing dialogue modeling methods have achieved promising performance o...

Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack

The detection of offensive language in the context of a dialogue has bec...

CloneBot: Personalized Dialogue-Response Predictions

Our project task was to create a model that, given a speaker ID, chat hi...