DeepSynth: Automata Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning

11/22/2019 ∙ by Mohammadhosein Hasanbeig, et al. ∙ 0

We propose a method for effective training of deep Reinforcement Learning (RL) agents when the reward is sparse and non-Markovian, but at the same time progress towards the reward requires the attainment of an unknown sequence of high-level objectives. Our method employs a recently-published algorithm for synthesis of compact automata to uncover this sequential structure. We synthesise an automaton from trace data generated through exploration of the environment by the deep RL agent. A product construction is then used to enrich the state space of the environment so that generation of an optimal control policy by deep RL is guided by the discovered structure encoded in the automaton. Our experiments show that our method is able to achieve training results that are otherwise difficult with state-of-the-art RL techniques unaided by external guidance.



There are no comments yet.


This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.