DeepAI AI Chat
Log In Sign Up

Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems

05/30/2022
by   Ting-En Lin, et al.
Alibaba Group
0

In this paper, we present Duplex Conversation, a multi-turn, multimodal spoken dialogue system that enables telephone-based agents to interact with customers like a human. We use the concept of full-duplex in telecommunication to demonstrate what a human-like interactive experience should be and how to achieve smooth turn-taking through three subtasks: user state detection, backchannel selection, and barge-in detection. Besides, we propose semi-supervised learning with multimodal data augmentation to leverage unlabeled data to increase model generalization. Experimental results on three sub-tasks show that the proposed method achieves consistent improvements compared with baselines. We deploy the Duplex Conversation to Alibaba intelligent customer service and share lessons learned in production. Online A/B experiments show that the proposed system can significantly reduce response latency by 50

READ FULL TEXT

page 1

page 5

10/30/2020

Improving Dialogue Breakdown Detection with Semi-Supervised Learning

Building user trust in dialogue agents requires smooth and consistent di...
05/18/2020

Neural Generation of Dialogue Response Timings

The timings of spoken response offsets in human dialogue have been shown...
09/14/2019

Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia

Dialogue technologies such as Amazon's Alexa have the potential to trans...
04/29/2019

A Persona-based Multi-turn Conversation Model in an Adversarial Learning Framework

In this paper, we extend the persona-based sequence-to-sequence (Seq2Seq...
04/18/2022

Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue

Turn-taking, aiming to decide when the next speaker can start talking, i...
01/14/2020

A Hybrid Solution to Learn Turn-Taking in Multi-Party Service-based Chat Groups

To predict the next most likely participant to interact in a multi-party...
05/02/2021

Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview

Following the success of spoken dialogue systems (SDS) in smartphone ass...