The Predictron: End-To-End Learning and Planning

12/28/2016
by   David Silver, et al.
0

One of the key challenges of artificial intelligence is to learn models that are effective in the context of planning. In this document we introduce the predictron architecture. The predictron consists of a fully abstract model, represented by a Markov reward process, that can be rolled forward multiple "imagined" planning steps. Each forward pass of the predictron accumulates internal rewards and values over multiple planning depths. The predictron is trained end-to-end so as to make these accumulated values accurately approximate the true value function. We applied the predictron to procedurally generated random mazes and a simulator for the game of pool. The predictron yielded significantly more accurate predictions than conventional deep neural network architectures.

READ FULL TEXT

page 4

page 11

02/09/2016

Value Iteration Networks

We introduce the value iteration network (VIN): a fully differentiable n...
11/19/2019

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Constructing agents with planning capabilities has long been one of the ...
10/31/2017

TreeQN and ATreeC: Differentiable Tree Planning for Deep Reinforcement Learning

Combining deep model-free reinforcement learning with on-line planning i...
12/16/2019

Planning with Abstract Learned Models While Learning Transferable Subtasks

We introduce an algorithm for model-based hierarchical reinforcement lea...
04/02/2018

Universal Planning Networks

A key challenge in complex visuomotor control is learning abstract repre...
03/20/2017

QMDP-Net: Deep Learning for Planning under Partial Observability

This paper introduces the QMDP-net, a neural network architecture for pl...

Code Repositories

predictron

WIP implementation of "The Predictron: End-To-End Learning and Planning" (http://arxiv.org/abs/1612.08810) in Chainer


view repo