Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize

by   Chandra Khatri, et al.

Building open domain conversational systems that allow users to have engaging conversations on topics of their choice is a challenging task. Alexa Prize was launched in 2016 to tackle the problem of achieving natural, sustained, coherent and engaging open-domain dialogs. In the second iteration of the competition in 2018, university teams advanced the state of the art by using context in dialog models, leveraging knowledge graphs for language understanding, handling complex utterances, building statistical and hierarchical dialog managers, and leveraging model-driven signals from user responses. The 2018 competition also included the provision of a suite of tools and models to the competitors including the CoBot (conversational bot) toolkit, topic and dialog act detection models, conversation evaluators, and a sensitive content detection model so that the competing teams could focus on building knowledge-rich, coherent and engaging multi-turn dialog systems. This paper outlines the advances developed by the university teams as well as the Alexa Prize team to achieve the common goal of advancing the science of Conversational AI. We address several key open-ended problems such as conversational speech recognition, open domain natural language understanding, commonsense reasoning, statistical dialog management, and dialog evaluation. These collaborative efforts have driven improved experiences by Alexa users to an average rating of 3.61, the median duration of 2 mins 18 seconds, and average turns to 14.6, increases of 14 of the 2018 competition. For conversational speech recognition, we have improved our relative Word Error Rate by 55 by 34 significantly more rapidly in 2018, in part due to the release of the CoBot toolkit.


Conversational AI: The Science Behind the Alexa Prize

Conversational agents are exploding in popularity. However, much work re...

Gunrock 2.0: A User Adaptive Social Conversational System

Gunrock 2.0 is built on top of Gunrock with an emphasis on user adaptati...

Audrey: A Personalized Open-Domain Conversational Bot

Conversational Intelligence requires that a person engage on information...

Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations

We present Chirpy Cardinal, an open-domain dialogue agent, as a research...

Building A User-Centric and Content-Driven Socialbot

To build Sounding Board, we develop a system architecture that is capabl...

A Probabilistic Framework for Representing Dialog Systems and Entropy-Based Dialog Management through Dynamic Stochastic State Evolution

In this paper, we present a probabilistic framework for goal-driven spok...

Entertaining and Opinionated but Too Controlling: A Large-Scale User Study of an Open Domain Alexa Prize System

Conversational systems typically focus on functional tasks such as sched...