Toward Interpretability of Dual-Encoder Models for Dialogue Response Suggestions

03/02/2020
by   Yitong Li, et al.
0

This work shows how to improve and interpret the commonly used dual encoder model for response suggestion in dialogue. We present an attentive dual encoder model that includes an attention mechanism on top of the extracted word-level features from two encoders, one for context and one for label respectively. To improve the interpretability in the dual encoder models, we design a novel regularization loss to minimize the mutual information between unimportant words and desired labels, in addition to the original attention method, so that important words are emphasized while unimportant words are de-emphasized. This can help not only with model interpretability, but can also further improve model accuracy. We propose an approximation method that uses a neural network to calculate the mutual information. Furthermore, by adding a residual layer between raw word embeddings and the final encoded context feature, word-level interpretability is preserved at the final prediction of the model. We compare the proposed model with existing methods for the dialogue response task on two public datasets (Persona and Ubuntu). The experiments demonstrate the effectiveness of the proposed model in terms of better Recall@1 accuracy and visualized interpretability.

READ FULL TEXT
research
07/28/2019

Hybrid Code Networks using a convolutional neural network as an input layer achieves higher turn accuracy

The dialogue management is a task of conversational artificial intellige...
research
10/01/2021

Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning

Establishing retrieval-based dialogue systems that can select appropriat...
research
03/21/2019

Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer

With the increasing research interest in dialogue response generation, t...
research
07/09/2019

ReCoSa: Detecting the Relevant Contexts with Self-Attention for Multi-turn Dialogue Generation

In multi-turn dialogue generation, response is usually related with only...
research
02/03/2023

Improving Interpretability via Explicit Word Interaction Graph Layer

Recent NLP literature has seen growing interest in improving model inter...
research
12/03/2018

Building Sequential Inference Models for End-to-End Response Selection

This paper presents an end-to-end response selection model for Track 1 o...
research
08/05/2021

M2IOSR: Maximal Mutual Information Open Set Recognition

In this work, we aim to address the challenging task of open set recogni...

Please sign up or login with your details

Forgot password? Click here to reset