Document-editing Assistants and Model-based Reinforcement Learning as a Path to Conversational AI

by   Katya Kudashkina, et al.

Intelligent assistants that follow commands or answer simple questions, such as Siri and Google search, are among the most economically important applications of AI. Future conversational AI assistants promise even greater capabilities and a better user experience through a deeper understanding of the domain, the user, or the user's purposes. But what domain and what methods are best suited to researching and realizing this promise? In this article we argue for the domain of voice document editing and for the methods of model-based reinforcement learning. The primary advantages of voice document editing are that the domain is tightly scoped and that it provides something for the conversation to be about (the document) that is delimited and fully accessible to the intelligent assistant. The advantages of reinforcement learning in general are that its methods are designed to learn from interaction without explicit instruction and that it formalizes the purposes of the assistant. Model-based reinforcement learning is needed in order to genuinely understand the domain of discourse and thereby work efficiently with the user to achieve their goals. Together, voice document editing and model-based reinforcement learning comprise a promising research direction for achieving conversational AI.


page 1

page 2

page 3

page 4


Deep Reinforcement Learning for Conversational AI

Deep reinforcement learning is revolutionizing the artificial intelligen...

Deep Reinforcement Learning for Dialogue Generation

Recent neural models of dialogue generation offer great promise for gene...

Using reinforcement learning to design an AI assistantfor a satisfying co-op experience

In this project, we designed an intelligent assistant player for the sin...

The Arcade Learning Environment: An Evaluation Platform for General Agents

In this article we introduce the Arcade Learning Environment (ALE): both...

Digital Einstein Experience: Fast Text-to-Speech for Conversational AI

We describe our approach to create and deliver a custom voice for a conv...

The RLLChatbot: a solution to the ConvAI Challenge

Current conversational systems can follow simple commands and answer bas...

Model-based Bayesian Reinforcement Learning for Dialogue Management

Reinforcement learning methods are increasingly used to optimise dialogu...