Structured Hierarchical Dialogue Policy with Graph Neural Networks

09/22/2020
by   Zhi Chen, et al.
0

Dialogue policy training for composite tasks, such as restaurant reservation in multiple places, is a practically important and challenging problem. Recently, hierarchical deep reinforcement learning (HDRL) methods have achieved good performance in composite tasks. However, in vanilla HDRL, both top-level and low-level policies are all represented by multi-layer perceptrons (MLPs) which take the concatenation of all observations from the environment as the input for predicting actions. Thus, traditional HDRL approach often suffers from low sampling efficiency and poor transferability. In this paper, we address these problems by utilizing the flexibility of graph neural networks (GNNs). A novel ComNet is proposed to model the structure of a hierarchical agent. The performance of ComNet is tested on composited tasks of the PyDial benchmark. Experiments show that ComNet outperforms vanilla HDRL systems with performance close to the upper bound. It not only achieves sample efficiency but also is more robust to noise while maintaining the transferability to other composite tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/10/2017

Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning

Building a dialogue agent to fulfill complex tasks, such as travel plann...
research
05/27/2019

AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning

Dialogue policy plays an important role in task-oriented spoken dialogue...
research
07/27/2020

Hierarchical BiGraph Neural Network as Recommendation Systems

Graph neural networks emerge as a promising modeling method for applicat...
research
04/20/2018

Subgoal Discovery for Hierarchical Dialogue Policy Learning

Developing conversational agents to engage in complex dialogues is chall...
research
04/09/2020

Recognizing Spatial Configurations of Objects with Graph Neural Networks

Deep learning algorithms can be seen as compositions of functions acting...
research
02/22/2023

Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues

Reinforcement learning has been widely adopted to model dialogue manager...
research
09/30/2019

Off-policy Multi-step Q-learning

In the past few years, off-policy reinforcement learning methods have sh...

Please sign up or login with your details

Forgot password? Click here to reset