DeepSynth: Program Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning

11/22/2019
by   Mohammadhosein Hasanbeig, et al.
0

We propose a method for efficient training of deep Reinforcement Learning (RL) agents when the reward is highly sparse and non-Markovian, but at the same time admits a high-level yet unknown sequential structure, as seen in a number of video games. This high-level sequential structure can be expressed as a computer program, which our method infers automatically as the RL agent explores the environment. Through this process, a high-level sequential task that occurs only rarely may nonetheless be encoded within the inferred program. A hybrid architecture for deep neural fitted Q-iteration is then employed to fill in low-level details and generate an optimal control policy that follows the structure of the program. Our experiments show that the agent is able to synthesise a complex program to guide the RL exploitation phase, which is otherwise difficult to achieve with state-of-the-art RL techniques.

READ FULL TEXT
research
11/22/2019

DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning

We propose a method for effective training of deep Reinforcement Learnin...
research
09/23/2019

Modular Deep Reinforcement Learning with Temporal Logic Specifications

We propose an actor-critic, model-free, and online Reinforcement Learnin...
research
03/25/2021

Hierarchical Program-Triggered Reinforcement Learning Agents For Automated Driving

Recent advances in Reinforcement Learning (RL) combined with Deep Learni...
research
06/17/2021

A Deep Reinforcement Learning Approach towards Pendulum Swing-up Problem based on TF-Agents

Adapting the idea of training CartPole with Deep Q-learning agent, we ar...
research
02/22/2021

Program Synthesis Guided Reinforcement Learning

A key challenge for reinforcement learning is solving long-horizon plann...
research
07/19/2018

FuzzerGym: A Competitive Framework for Fuzzing and Learning

Fuzzing is a commonly used technique designed to test software by automa...
research
10/03/2022

Mastering Spatial Graph Prediction of Road Networks

Accurately predicting road networks from satellite images requires a glo...

Please sign up or login with your details

Forgot password? Click here to reset