Time to Transfer: Predicting and Evaluating Machine-Human Chatting Handoff

12/14/2020
by   Jiawei Liu, et al.
0

Is chatbot able to completely replace the human agent? The short answer could be - "it depends...". For some challenging cases, e.g., dialogue's topical spectrum spreads beyond the training corpus coverage, the chatbot may malfunction and return unsatisfied utterances. This problem can be addressed by introducing the Machine-Human Chatting Handoff (MHCH), which enables human-algorithm collaboration. To detect the normal/transferable utterances, we propose a Difficulty-Assisted Matching Inference (DAMI) network, utilizing difficulty-assisted encoding to enhance the representations of utterances. Moreover, a matching inference mechanism is introduced to capture the contextual matching features. A new evaluation metric, Golden Transfer within Tolerance (GT-T), is proposed to assess the performance by considering the tolerance property of the MHCH. To provide insights into the task and validate the proposed model, we collect two new datasets. Extensive experimental results are presented and contrasted against a series of baseline models to demonstrate the efficacy of our model on MHCH.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/01/2020

Learning an Unreferenced Metric for Online Dialogue Evaluation

Evaluating the quality of a dialogue interaction between two agents is a...
research
10/24/2022

Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?

Task-oriented dialogue (TOD) systems have assisted users on many tasks, ...
research
02/10/2021

Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge

Multi-turn dialogue reading comprehension aims to teach machines to read...
research
06/06/2023

Revisiting Conversation Discourse for Dialogue Disentanglement

Dialogue disentanglement aims to detach the chronologically ordered utte...
research
05/16/2018

Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks

Recent approaches for dialogue act recognition have shown that context f...
research
10/06/2020

Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection

Few-shot Intent Detection is challenging due to the scarcity of availabl...
research
11/10/2019

Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training

Generative dialogue models currently suffer from a number of problems wh...

Please sign up or login with your details

Forgot password? Click here to reset