Time to Transfer: Predicting and Evaluating Machine-Human Chatting Handoff

by   Jiawei Liu, et al.

Is chatbot able to completely replace the human agent? The short answer could be - "it depends...". For some challenging cases, e.g., dialogue's topical spectrum spreads beyond the training corpus coverage, the chatbot may malfunction and return unsatisfied utterances. This problem can be addressed by introducing the Machine-Human Chatting Handoff (MHCH), which enables human-algorithm collaboration. To detect the normal/transferable utterances, we propose a Difficulty-Assisted Matching Inference (DAMI) network, utilizing difficulty-assisted encoding to enhance the representations of utterances. Moreover, a matching inference mechanism is introduced to capture the contextual matching features. A new evaluation metric, Golden Transfer within Tolerance (GT-T), is proposed to assess the performance by considering the tolerance property of the MHCH. To provide insights into the task and validate the proposed model, we collect two new datasets. Extensive experimental results are presented and contrasted against a series of baseline models to demonstrate the efficacy of our model on MHCH.


page 1

page 2

page 3

page 4


Learning an Unreferenced Metric for Online Dialogue Evaluation

Evaluating the quality of a dialogue interaction between two agents is a...

Are Current Task-oriented Dialogue Systems Able to Satisfy Impolite Users?

Task-oriented dialogue (TOD) systems have assisted users on many tasks, ...

Multi-turn Dialogue Reading Comprehension with Pivot Turns and Knowledge

Multi-turn dialogue reading comprehension aims to teach machines to read...

Revisiting Conversation Discourse for Dialogue Disentanglement

Dialogue disentanglement aims to detach the chronologically ordered utte...

Conversational Analysis using Utterance-level Attention-based Bidirectional Recurrent Neural Networks

Recent approaches for dialogue act recognition have shown that context f...

Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection

Few-shot Intent Detection is challenging due to the scarcity of availabl...

Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training

Generative dialogue models currently suffer from a number of problems wh...

Please sign up or login with your details

Forgot password? Click here to reset