Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning

04/10/2017
by   Baolin Peng, et al.
0

Building a dialogue agent to fulfill complex tasks, such as travel planning, is challenging because the agent has to learn to collectively complete multiple subtasks. For example, the agent needs to reserve a hotel and book a flight so that there leaves enough time for commute between arrival and hotel check-in. This paper addresses this challenge by formulating the task in the mathematical framework of options over Markov Decision Processes (MDPs), and proposing a hierarchical deep reinforcement learning approach to learning a dialogue manager that operates at different temporal scales. The dialogue manager consists of: (1) a top-level dialogue policy that selects among subtasks or options, (2) a low-level dialogue policy that selects primitive actions to complete the subtask given by the top-level policy, and (3) a global state tracker that helps ensure all cross-subtask constraints be satisfied. Experiments on a travel planning task with simulated and real users show that our approach leads to significant improvements over three baselines, two based on handcrafted rules and the other based on flat deep reinforcement learning.

READ FULL TEXT

page 3

page 5

page 7

research
04/20/2018

Subgoal Discovery for Hierarchical Dialogue Policy Learning

Developing conversational agents to engage in complex dialogues is chall...
research
09/22/2020

Structured Hierarchical Dialogue Policy with Graph Neural Networks

Dialogue policy training for composite tasks, such as restaurant reserva...
research
01/18/2018

Integrating planning for task-completion dialogue policy learning

Training a task-completion dialogue agent with real users via reinforcem...
research
05/27/2019

AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning

Dialogue policy plays an important role in task-oriented spoken dialogue...
research
06/19/2017

Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning

Human conversation is inherently complex, often spanning many different ...
research
06/02/2019

Budgeted Policy Learning for Task-Oriented Dialogue Systems

This paper presents a new approach that extends Deep Dyna-Q (DDQ) by inc...
research
08/28/2018

Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning

This paper presents a Discriminative Deep Dyna-Q (D3Q) approach to impro...

Please sign up or login with your details

Forgot password? Click here to reset