DeepAI AI Chat
Log In Sign Up

Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systems

by   Ting-En Lin, et al.
Alibaba Group

In this paper, we present Duplex Conversation, a multi-turn, multimodal spoken dialogue system that enables telephone-based agents to interact with customers like a human. We use the concept of full-duplex in telecommunication to demonstrate what a human-like interactive experience should be and how to achieve smooth turn-taking through three subtasks: user state detection, backchannel selection, and barge-in detection. Besides, we propose semi-supervised learning with multimodal data augmentation to leverage unlabeled data to increase model generalization. Experimental results on three sub-tasks show that the proposed method achieves consistent improvements compared with baselines. We deploy the Duplex Conversation to Alibaba intelligent customer service and share lessons learned in production. Online A/B experiments show that the proposed system can significantly reduce response latency by 50


page 1

page 5


Improving Dialogue Breakdown Detection with Semi-Supervised Learning

Building user trust in dialogue agents requires smooth and consistent di...

Neural Generation of Dialogue Response Timings

The timings of spoken response offsets in human dialogue have been shown...

Current Challenges in Spoken Dialogue Systems and Why They Are Critical for Those Living with Dementia

Dialogue technologies such as Amazon's Alexa have the potential to trans...

A Persona-based Multi-turn Conversation Model in an Adversarial Learning Framework

In this paper, we extend the persona-based sequence-to-sequence (Seq2Seq...

Gated Multimodal Fusion with Contrastive Learning for Turn-taking Prediction in Human-robot Dialogue

Turn-taking, aiming to decide when the next speaker can start talking, i...

A Hybrid Solution to Learn Turn-Taking in Multi-Party Service-based Chat Groups

To predict the next most likely participant to interact in a multi-party...

Intelligent Conversational Android ERICA Applied to Attentive Listening and Job Interview

Following the success of spoken dialogue systems (SDS) in smartphone ass...