Time to Transfer: Predicting and Evaluating Machine-Human Chatting Handoff

12/14/2020
by   Jiawei Liu, et al.
0

Is chatbot able to completely replace the human agent? The short answer could be - "it depends...". For some challenging cases, e.g., dialogue's topical spectrum spreads beyond the training corpus coverage, the chatbot may malfunction and return unsatisfied utterances. This problem can be addressed by introducing the Machine-Human Chatting Handoff (MHCH), which enables human-algorithm collaboration. To detect the normal/transferable utterances, we propose a Difficulty-Assisted Matching Inference (DAMI) network, utilizing difficulty-assisted encoding to enhance the representations of utterances. Moreover, a matching inference mechanism is introduced to capture the contextual matching features. A new evaluation metric, Golden Transfer within Tolerance (GT-T), is proposed to assess the performance by considering the tolerance property of the MHCH. To provide insights into the task and validate the proposed model, we collect two new datasets. Extensive experimental results are presented and contrasted against a series of baseline models to demonstrate the efficacy of our model on MHCH.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

05/01/2020

Learning an Unreferenced Metric for Online Dialogue Evaluation

Evaluating the quality of a dialogue interaction between two agents is a...
02/10/2021

Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge

Multi-turn dialogue reading comprehension aims to teach machines to read...
07/16/2021

Darmok and Jalad at Tanagra: A Dataset and Model for English-to-Tamarian Translation

Tamarian, a fictional language introduced in the Star Trek episode Darmo...
05/16/2018

Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks

Recent approaches for dialogue act recognition have shown that context f...
10/06/2020

Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection

Few-shot Intent Detection is challenging due to the scarcity of availabl...
12/20/2016

Unsupervised Dialogue Act Induction using Gaussian Mixtures

This paper introduces a new unsupervised approach for dialogue act induc...
11/10/2019

Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training

Generative dialogue models currently suffer from a number of problems wh...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.